[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:37 dist-test-slave-k5rr krb5kdc[9172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:37 dist-test-slave-k5rr krb5kdc[9172](info): set up 2 sockets
Aug 06 18:39:37 dist-test-slave-k5rr krb5kdc[9172](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 18:39:39.078518 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:39:39 dist-test-slave-k5rr krb5kdc[9172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:39:39.209615 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37003
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:37003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:39.325560 9188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:39.325862 9188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:39.325920 9188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:39.329957 9188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:39.330027 9188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:39.330055 9188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:39.330077 9188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:39.330106 9188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:39.335561 9188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:37003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:39.336685 9188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:39.337787 9188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:39:39.346851 9188 server_base.cc:1034] running on GCE node
W20260806 18:39:39.346788 9196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:39.346797 9201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:39.346825 9203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:39.347630 9188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:39.348771 9188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:39.349929 9188 hybrid_clock.cc:648] HybridClock initialized: now 1786041579349926 us; error 107 us; skew 500 ppm
Aug 06 18:39:39 dist-test-slave-k5rr krb5kdc[9172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:39.352694 9188 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:39:39.353654 9188 webserver.cc:460] Webserver started at http://127.8.241.254:46351/ using document root <none> and password file <none>
I20260806 18:39:39.354211 9188 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:39.354281 9188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:39.354498 9188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:39.356712 9188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd"
format_stamp: "Formatted at 2026-08-06 18:39:39 on dist-test-slave-k5rr"
server_key: "f8b3dd6c2aaa4fd6c030c3c318e1eba8"
server_key_iv: "98a66344bb6d278af5b7de46f4b40a99"
server_key_version: "encryptionkey@0"
I20260806 18:39:39.357331 9188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd"
format_stamp: "Formatted at 2026-08-06 18:39:39 on dist-test-slave-k5rr"
server_key: "f8b3dd6c2aaa4fd6c030c3c318e1eba8"
server_key_iv: "98a66344bb6d278af5b7de46f4b40a99"
server_key_version: "encryptionkey@0"
I20260806 18:39:39.361061 9188 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:39:39.363397 9212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:39.364598 9188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:39:39.364712 9188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd"
format_stamp: "Formatted at 2026-08-06 18:39:39 on dist-test-slave-k5rr"
server_key: "f8b3dd6c2aaa4fd6c030c3c318e1eba8"
server_key_iv: "98a66344bb6d278af5b7de46f4b40a99"
server_key_version: "encryptionkey@0"
I20260806 18:39:39.364841 9188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:39.395020 9188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:39.397979 9188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:39.398164 9188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:39.405408 9188 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:37003
I20260806 18:39:39.405432 9285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:37003 every 8 connection(s)
I20260806 18:39:39.406390 9188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:39.407286 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9188
I20260806 18:39:39.407429 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:39:39.408939 9159 external_mini_cluster.cc:1428] Setting key d299f746008065fcea1ae9e932cbc182
I20260806 18:39:39.409359 9286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:39.415611 9286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd: Bootstrap starting.
I20260806 18:39:39.418131 9286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:39.418850 9286 log.cc:824] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:39.420730 9286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd: No bootstrap required, opened a new log
I20260806 18:39:39.424151 9286 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37003 } }
I20260806 18:39:39.424399 9286 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:39.424463 9286 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c7f8c8d7e1d497a9e9e39a74a3f1dcd, State: Initialized, Role: FOLLOWER
I20260806 18:39:39.424883 9286 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37003 } }
I20260806 18:39:39.425035 9286 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:39.425105 9286 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:39.425203 9286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [term 0 FOLLOWER]: Advancing to term 1
Aug 06 18:39:39 dist-test-slave-k5rr krb5kdc[9172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:39.426198 9286 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37003 } }
I20260806 18:39:39.426517 9286 leader_election.cc:302] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c7f8c8d7e1d497a9e9e39a74a3f1dcd; no voters:
I20260806 18:39:39.426784 9286 leader_election.cc:288] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:39.426957 9296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:39.427193 9296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [term 1 LEADER]: Becoming Leader. State: Replica: 9c7f8c8d7e1d497a9e9e39a74a3f1dcd, State: Running, Role: LEADER
I20260806 18:39:39.427493 9296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37003 } }
I20260806 18:39:39.428126 9286 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:39.429245 9298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c7f8c8d7e1d497a9e9e39a74a3f1dcd. Latest consensus state: current_term: 1 leader_uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37003 } } }
I20260806 18:39:39.429384 9298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:39.429692 9297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c7f8c8d7e1d497a9e9e39a74a3f1dcd" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37003 } } }
I20260806 18:39:39.429813 9297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:39.429894 9309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:39.433405 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:39.412143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51560 (local address 127.8.241.254:37003)
0806 18:39:39.412552 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:39.412567 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:39.413707 (+ 1140us) server_negotiation.cc:407] Connection header received
0806 18:39:39.416743 (+ 3036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:39.416765 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:39.417225 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:39.417592 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:39.420151 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.421200 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.422353 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.422811 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.422955 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:39.427092 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:39.427098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:39.427114 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:39.427146 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:39.429244 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:39.429924 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:39.429930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:39.429936 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:39.430078 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:39.430517 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:39.430520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:39.430522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:39.431057 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:39.431257 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:39.432146 (+ 889us) server_negotiation.cc:299] Negotiation successful
0806 18:39:39.432446 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":120,"threads_started":1}
I20260806 18:39:39.433846 9309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:39.440554 9309 catalog_manager.cc:1347] Generated new cluster ID: 60d0f5c0f74c476fadd370cb4cbf1811
I20260806 18:39:39.440619 9309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:39.460736 9309 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:39.462194 9309 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:39.475158 9309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c7f8c8d7e1d497a9e9e39a74a3f1dcd: Generated new TSK 0
I20260806 18:39:39.476105 9309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:39:39.513741 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37003
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:39.634079 9324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:39.634392 9324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:39.634473 9324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:39.638705 9324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:39.638788 9324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:39.638875 9324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:39:39.645030 9324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37003
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:39.646261 9324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:39.647405 9324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:39.658648 9330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:39.658735 9334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:39.658838 9324 server_base.cc:1034] running on GCE node
W20260806 18:39:39.659015 9332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:39.659493 9324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:39.660192 9324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:39.661422 9324 hybrid_clock.cc:648] HybridClock initialized: now 1786041579661399 us; error 39 us; skew 500 ppm
Aug 06 18:39:39 dist-test-slave-k5rr krb5kdc[9172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:39.664569 9324 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:39:39.665630 9324 webserver.cc:460] Webserver started at http://127.8.241.193:35185/ using document root <none> and password file <none>
I20260806 18:39:39.666234 9324 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:39.666301 9324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:39.666522 9324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:39.669041 9324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "228d5e73bd7f4bba9cf752ffe4af4d0a"
format_stamp: "Formatted at 2026-08-06 18:39:39 on dist-test-slave-k5rr"
server_key: "ddbdc442d3ef8bfeb7447d02b5dfba19"
server_key_iv: "eceb7fa6d0a353fa4a1195d4d37b5d5d"
server_key_version: "encryptionkey@0"
I20260806 18:39:39.669720 9324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "228d5e73bd7f4bba9cf752ffe4af4d0a"
format_stamp: "Formatted at 2026-08-06 18:39:39 on dist-test-slave-k5rr"
server_key: "ddbdc442d3ef8bfeb7447d02b5dfba19"
server_key_iv: "eceb7fa6d0a353fa4a1195d4d37b5d5d"
server_key_version: "encryptionkey@0"
I20260806 18:39:39.674008 9324 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:39:39.676680 9346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:39.678030 9324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:39:39.678159 9324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "228d5e73bd7f4bba9cf752ffe4af4d0a"
format_stamp: "Formatted at 2026-08-06 18:39:39 on dist-test-slave-k5rr"
server_key: "ddbdc442d3ef8bfeb7447d02b5dfba19"
server_key_iv: "eceb7fa6d0a353fa4a1195d4d37b5d5d"
server_key_version: "encryptionkey@0"
I20260806 18:39:39.678282 9324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:39.699496 9324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:39.702809 9324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:39.702997 9324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:39.703636 9324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:39.704563 9324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:39.704610 9324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:39.704654 9324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:39.704682 9324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:39.714650 9324 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:34591
I20260806 18:39:39.714679 9515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:34591 every 8 connection(s)
I20260806 18:39:39.715642 9324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:39:39.721544 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9324
I20260806 18:39:39.721657 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:39:39.721964 9159 external_mini_cluster.cc:1428] Setting key f797ee68f9c5a1d49d6e57289ff59033
Aug 06 18:39:39 dist-test-slave-k5rr krb5kdc[9172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:39.730180 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:39.717680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:42403 (local address 127.8.241.254:37003)
0806 18:39:39.717825 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:39.717829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:39.718727 (+ 898us) server_negotiation.cc:407] Connection header received
0806 18:39:39.720083 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:39.720086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:39.720135 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:39.720206 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:39.721457 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.722228 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.723415 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.723662 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.723754 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:39.726300 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:39.726303 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:39.726306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:39.726337 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:39.727883 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:39.728427 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:39.728431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:39.728433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:39.728544 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:39.728981 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:39.728984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:39.728986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:39.729246 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:39.729355 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:39.729863 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 18:39:39.730006 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:39:39.731017 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:39.718085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37003 (local address 127.8.241.193:42403)
0806 18:39:39.718572 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:39.718617 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:39.719866 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:39.720363 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:39.720371 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:39.720725 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:39.721308 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:39.721324 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.722359 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.722362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:39.723286 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:39.723296 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.723810 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.723814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:39.723997 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:39.724564 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:39.724580 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:39.726116 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:39.728006 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:39.728012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:39.728025 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:39.728302 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:39.728635 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:39.728637 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:39.728639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:39.728894 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:39.729348 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:39.729353 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:39.729603 (+ 250us) client_negotiation.cc:769] Sending connection context
0806 18:39:39.729801 (+ 198us) client_negotiation.cc:240] Negotiation successful
0806 18:39:39.730123 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":126,"threads_started":1}
I20260806 18:39:39.732395 9516 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37003
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:39:39.732683 9516 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:39.733296 9516 heartbeater.cc:511] Master 127.8.241.254:37003 requested a full tablet report, sending...
I20260806 18:39:39.734936 9235 ts_manager.cc:194] Registered new tserver with Master: 228d5e73bd7f4bba9cf752ffe4af4d0a (127.8.241.193:34591)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:39:39.736325 9235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:42403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:39:39.750274 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37003
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:39.875454 9524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:39.875758 9524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:39.875805 9524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:39.879956 9524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:39.880038 9524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:39.880100 9524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:39:39.885795 9524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37003
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:39.887041 9524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:39.888123 9524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:39.898769 9532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:39.898881 9524 server_base.cc:1034] running on GCE node
W20260806 18:39:39.898763 9531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:39.898758 9534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:39.899495 9524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:39.900226 9524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:39.901427 9524 hybrid_clock.cc:648] HybridClock initialized: now 1786041579901400 us; error 38 us; skew 500 ppm
Aug 06 18:39:39 dist-test-slave-k5rr krb5kdc[9172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:39.904336 9524 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:39:39.905316 9524 webserver.cc:460] Webserver started at http://127.8.241.194:38445/ using document root <none> and password file <none>
I20260806 18:39:39.905885 9524 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:39.905946 9524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:39.906123 9524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:39.908320 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "5fed9211a4a441cdab85f60f394348c3"
format_stamp: "Formatted at 2026-08-06 18:39:39 on dist-test-slave-k5rr"
server_key: "7bc3edf9114f5c8671bd20e0a642d5af"
server_key_iv: "3f76df53f6eefe51f34167bc5387f3fc"
server_key_version: "encryptionkey@0"
I20260806 18:39:39.908874 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "5fed9211a4a441cdab85f60f394348c3"
format_stamp: "Formatted at 2026-08-06 18:39:39 on dist-test-slave-k5rr"
server_key: "7bc3edf9114f5c8671bd20e0a642d5af"
server_key_iv: "3f76df53f6eefe51f34167bc5387f3fc"
server_key_version: "encryptionkey@0"
I20260806 18:39:39.912959 9524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 18:39:39.915693 9546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:39.917137 9524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:39.917291 9524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "5fed9211a4a441cdab85f60f394348c3"
format_stamp: "Formatted at 2026-08-06 18:39:39 on dist-test-slave-k5rr"
server_key: "7bc3edf9114f5c8671bd20e0a642d5af"
server_key_iv: "3f76df53f6eefe51f34167bc5387f3fc"
server_key_version: "encryptionkey@0"
I20260806 18:39:39.917408 9524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:39.933245 9524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:39.936571 9524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:39.936759 9524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:39.937469 9524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:39.938380 9524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:39.938428 9524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:39.938482 9524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:39.938510 9524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:39.947484 9524 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:39877
I20260806 18:39:39.947516 9713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:39877 every 8 connection(s)
I20260806 18:39:39.948493 9524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:39:39.957917 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9524
I20260806 18:39:39.958029 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:39:39.958276 9159 external_mini_cluster.cc:1428] Setting key 51e9c7d33b6576ac5b970aca8c68ff85
Aug 06 18:39:39 dist-test-slave-k5rr krb5kdc[9172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:39.963533 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:39.950560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:60315 (local address 127.8.241.254:37003)
0806 18:39:39.950698 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:39.950702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:39.951720 (+ 1018us) server_negotiation.cc:407] Connection header received
0806 18:39:39.953251 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:39.953253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:39.953328 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:39.953405 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:39.954667 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.955195 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.956383 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.956591 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.956661 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:39.959538 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:39.959541 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:39.959543 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:39.959569 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:39.961084 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:39.961712 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:39.961715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:39.961717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:39.961818 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:39.962290 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:39.962293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:39.962294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:39.962521 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:39.962609 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:39.963250 (+ 641us) server_negotiation.cc:299] Negotiation successful
0806 18:39:39.963362 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:39:39.964550 9716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:39.950999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37003 (local address 127.8.241.194:60315)
0806 18:39:39.951541 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:39.951593 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:39.952964 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:39.953589 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:39.953597 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:39.954021 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:39.954552 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:39.954566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.955332 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.955335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:39.956283 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:39.956295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:39.956711 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:39.956713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:39.956922 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:39.957558 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:39.957576 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:39.959372 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:39.961281 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:39.961287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:39.961303 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:39.961602 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:39.961948 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:39.961951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:39.961954 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:39.962166 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:39.962627 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:39.962634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:39.962996 (+ 362us) client_negotiation.cc:769] Sending connection context
0806 18:39:39.963238 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 18:39:39.963603 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":116,"threads_started":1}
I20260806 18:39:39.965811 9714 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37003
I20260806 18:39:39.966066 9714 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:39.966645 9714 heartbeater.cc:511] Master 127.8.241.254:37003 requested a full tablet report, sending...
I20260806 18:39:39.967829 9235 ts_manager.cc:194] Registered new tserver with Master: 5fed9211a4a441cdab85f60f394348c3 (127.8.241.194:39877)
I20260806 18:39:39.968648 9235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:60315
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:39:39.988193 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37003
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:40.114042 9719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:40.114356 9719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:40.114405 9719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:40.118616 9719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:40.118687 9719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:40.118748 9719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:39:40.124253 9719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37003
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:40.125447 9719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:40.126479 9719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:40.136453 9725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:40.136453 9726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:40.136453 9729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:40.136847 9719 server_base.cc:1034] running on GCE node
I20260806 18:39:40.137318 9719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:40.137961 9719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:40.139118 9719 hybrid_clock.cc:648] HybridClock initialized: now 1786041580139087 us; error 46 us; skew 500 ppm
Aug 06 18:39:40 dist-test-slave-k5rr krb5kdc[9172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:40.141897 9719 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:39:40.142834 9719 webserver.cc:460] Webserver started at http://127.8.241.195:44613/ using document root <none> and password file <none>
I20260806 18:39:40.143420 9719 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:40.143489 9719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:40.143703 9719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:40.145921 9719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "783d2735d0fa47e29c58a1b5f8bbf04c"
format_stamp: "Formatted at 2026-08-06 18:39:40 on dist-test-slave-k5rr"
server_key: "defc26dcbaee9d97ef8f34c83b8a15a1"
server_key_iv: "22ed8fb5aacc0c98ed1e2255a5be367b"
server_key_version: "encryptionkey@0"
I20260806 18:39:40.146479 9719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "783d2735d0fa47e29c58a1b5f8bbf04c"
format_stamp: "Formatted at 2026-08-06 18:39:40 on dist-test-slave-k5rr"
server_key: "defc26dcbaee9d97ef8f34c83b8a15a1"
server_key_iv: "22ed8fb5aacc0c98ed1e2255a5be367b"
server_key_version: "encryptionkey@0"
I20260806 18:39:40.150219 9719 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:39:40.152740 9739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:40.154021 9719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:39:40.154165 9719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "783d2735d0fa47e29c58a1b5f8bbf04c"
format_stamp: "Formatted at 2026-08-06 18:39:40 on dist-test-slave-k5rr"
server_key: "defc26dcbaee9d97ef8f34c83b8a15a1"
server_key_iv: "22ed8fb5aacc0c98ed1e2255a5be367b"
server_key_version: "encryptionkey@0"
I20260806 18:39:40.154268 9719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:40.172185 9719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:40.175256 9719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:40.175444 9719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:40.176137 9719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:40.177181 9719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:40.177254 9719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:40.177304 9719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:40.177328 9719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:40.187062 9719 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:34633
I20260806 18:39:40.187086 9892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:34633 every 8 connection(s)
I20260806 18:39:40.187986 9719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:39:40.196261 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9719
I20260806 18:39:40.196378 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:39:40.196633 9159 external_mini_cluster.cc:1428] Setting key f4d60cf690c4b7bdc5a51ee211a03f8b
Aug 06 18:39:40 dist-test-slave-k5rr krb5kdc[9172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:40.201642 9288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:40.190005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:37267 (local address 127.8.241.254:37003)
0806 18:39:40.190159 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:40.190165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:40.191037 (+ 872us) server_negotiation.cc:407] Connection header received
0806 18:39:40.192427 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:40.192431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:40.192491 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:40.192565 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:40.193806 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:40.194329 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:40.195428 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:40.195646 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:40.195715 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:40.198218 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:40.198222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:40.198225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:40.198251 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:40.199743 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:40.200244 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:40.200247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:40.200248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:40.200329 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:40.200591 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:40.200593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:40.200594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:40.200796 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:40.200872 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:40.201362 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 18:39:40.201460 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:39:40.202437 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:40.190424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37003 (local address 127.8.241.195:37267)
0806 18:39:40.190883 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:40.190927 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:40.192184 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:40.192724 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:40.192731 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:40.193144 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:40.193675 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:40.193691 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:40.194432 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:40.194434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:40.195330 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:40.195339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:40.195753 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:40.195756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:40.195890 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:40.196474 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:40.196495 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:40.198064 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:40.199890 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:40.199895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:40.199908 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:40.200160 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:40.200395 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:40.200397 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:40.200399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:40.200529 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:40.200870 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:40.200875 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:40.201162 (+ 287us) client_negotiation.cc:769] Sending connection context
0806 18:39:40.201321 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 18:39:40.201595 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260806 18:39:40.203584 9894 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37003
I20260806 18:39:40.203821 9894 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:40.204327 9894 heartbeater.cc:511] Master 127.8.241.254:37003 requested a full tablet report, sending...
I20260806 18:39:40.205336 9235 ts_manager.cc:194] Registered new tserver with Master: 783d2735d0fa47e29c58a1b5f8bbf04c (127.8.241.195:34633)
I20260806 18:39:40.205864 9235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:37267
I20260806 18:39:40.210992 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:39:40 dist-test-slave-k5rr krb5kdc[9172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041580, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:39:40 dist-test-slave-k5rr krb5kdc[9172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041580, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:39:40.238009 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:40.228144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.8.241.193:34591)
0806 18:39:40.228463 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:40.228478 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:40.228504 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:39:40.228594 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:40.228599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:40.228905 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:40.229076 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:40.229719 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:40.230577 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:40.231400 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:40.231902 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:40.232000 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:40.234359 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:40.234371 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:40.234387 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:40.234417 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:40.236108 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:40.236517 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:40.236524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:40.236526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:40.236620 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:40.236874 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:40.236878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:40.236880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:40.237250 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:40.237451 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:40.237621 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 18:39:40.237797 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":96,"threads_started":1}
W20260806 18:39:40.239110 9385 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47410
Aug 06 18:39:40 dist-test-slave-k5rr krb5kdc[9172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:39:40 dist-test-slave-k5rr krb5kdc[9172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:39:40.266283 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:40.257745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.8.241.193:34591)
0806 18:39:40.257879 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:40.257884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:40.258149 (+ 265us) server_negotiation.cc:407] Connection header received
0806 18:39:40.258296 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:40.258300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:40.258351 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:40.258422 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:40.259204 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:40.259740 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:40.260582 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:40.260827 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:40.260920 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:40.263418 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:40.263421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:40.263423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:40.263459 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:40.264670 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:40.265106 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:40.265110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:40.265111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:40.265190 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:40.265442 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:40.265446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:40.265455 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:40.265634 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:40.265755 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:40.265996 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 18:39:40.266118 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:39:40.267366 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9324
I20260806 18:39:40.282263 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9524
I20260806 18:39:40.307296 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9719
I20260806 18:39:40.321918 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9188
2026-08-06T18:39:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2571 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:40 dist-test-slave-k5rr krb5kdc[9917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:40 dist-test-slave-k5rr krb5kdc[9917](info): set up 2 sockets
Aug 06 18:39:40 dist-test-slave-k5rr krb5kdc[9917](info): commencing operation
krb5kdc: starting...
W20260806 18:39:41.485234 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:39:41 dist-test-slave-k5rr krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:39:41.610143 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:44397
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:44397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:41.724079 9935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:41.724349 9935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:41.724387 9935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:41.728498 9935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:41.728564 9935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:41.728582 9935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:41.728597 9935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:41.728611 9935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:41.733938 9935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:44397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:44397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:41.734980 9935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:41.736047 9935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:41.744534 9940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:41.744678 9935 server_base.cc:1034] running on GCE node
W20260806 18:39:41.744534 9945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:41.744858 9941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:41.745508 9935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:41.746637 9935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:41.747808 9935 hybrid_clock.cc:648] HybridClock initialized: now 1786041581747777 us; error 41 us; skew 500 ppm
Aug 06 18:39:41 dist-test-slave-k5rr krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041581, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:41.750533 9935 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:39:41.751449 9935 webserver.cc:460] Webserver started at http://127.8.241.254:42947/ using document root <none> and password file <none>
I20260806 18:39:41.752046 9935 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:41.752110 9935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:41.752310 9935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:41.754527 9935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "9b63bab267a04f9083b9ee2003ae4d00"
format_stamp: "Formatted at 2026-08-06 18:39:41 on dist-test-slave-k5rr"
server_key: "1131952730a0fc7145fa7453274c6bea"
server_key_iv: "3b9725e8b9a28cab20726ba4278e2858"
server_key_version: "encryptionkey@0"
I20260806 18:39:41.755119 9935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "9b63bab267a04f9083b9ee2003ae4d00"
format_stamp: "Formatted at 2026-08-06 18:39:41 on dist-test-slave-k5rr"
server_key: "1131952730a0fc7145fa7453274c6bea"
server_key_iv: "3b9725e8b9a28cab20726ba4278e2858"
server_key_version: "encryptionkey@0"
I20260806 18:39:41.759186 9935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:39:41.761693 9953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:41.762915 9935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:39:41.763044 9935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "9b63bab267a04f9083b9ee2003ae4d00"
format_stamp: "Formatted at 2026-08-06 18:39:41 on dist-test-slave-k5rr"
server_key: "1131952730a0fc7145fa7453274c6bea"
server_key_iv: "3b9725e8b9a28cab20726ba4278e2858"
server_key_version: "encryptionkey@0"
I20260806 18:39:41.763151 9935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:41.789837 9935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:41.792872 9935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:41.793105 9935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:41.800247 9935 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:44397
I20260806 18:39:41.800261 10028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:44397 every 8 connection(s)
I20260806 18:39:41.801271 9935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:41.804186 10029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:41.806948 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9935
I20260806 18:39:41.807063 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:39:41.807338 9159 external_mini_cluster.cc:1428] Setting key 3b1bbf0d1a8ad65b6fd05e790d6641c0
I20260806 18:39:41.810608 10029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00: Bootstrap starting.
I20260806 18:39:41.812894 10029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:41.813665 10029 log.cc:824] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:41.815632 10029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00: No bootstrap required, opened a new log
Aug 06 18:39:41 dist-test-slave-k5rr krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:41.819620 10029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b63bab267a04f9083b9ee2003ae4d00" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44397 } }
I20260806 18:39:41.819897 10029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:41.819970 10029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b63bab267a04f9083b9ee2003ae4d00, State: Initialized, Role: FOLLOWER
I20260806 18:39:41.820561 10029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b63bab267a04f9083b9ee2003ae4d00" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44397 } }
I20260806 18:39:41.820736 10029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:41.820811 10029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:41.820922 10029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:41.822093 10029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b63bab267a04f9083b9ee2003ae4d00" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44397 } }
I20260806 18:39:41.822487 10029 leader_election.cc:302] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b63bab267a04f9083b9ee2003ae4d00; no voters:
I20260806 18:39:41.822638 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:41.808848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50704 (local address 127.8.241.254:44397)
0806 18:39:41.809320 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:41.809336 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:41.809374 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:39:41.810021 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:41.810039 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:41.810380 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:41.810706 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:41.811529 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:41.812534 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:41.813463 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:41.813972 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:41.814133 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:41.816671 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:41.816677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:41.816694 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:41.816725 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:41.819351 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:41.819776 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:41.819789 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:41.819796 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:41.819940 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:41.820221 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:41.820224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:41.820225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:41.820750 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:41.820969 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:41.821273 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 18:39:41.821584 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":160,"threads_started":1}
I20260806 18:39:41.822870 10029 leader_election.cc:288] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:41.822974 10034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:41.823194 10034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [term 1 LEADER]: Becoming Leader. State: Replica: 9b63bab267a04f9083b9ee2003ae4d00, State: Running, Role: LEADER
I20260806 18:39:41.823609 10034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b63bab267a04f9083b9ee2003ae4d00" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44397 } }
I20260806 18:39:41.824234 10029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:41.825495 10036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b63bab267a04f9083b9ee2003ae4d00. Latest consensus state: current_term: 1 leader_uuid: "9b63bab267a04f9083b9ee2003ae4d00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b63bab267a04f9083b9ee2003ae4d00" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44397 } } }
I20260806 18:39:41.825493 10035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b63bab267a04f9083b9ee2003ae4d00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b63bab267a04f9083b9ee2003ae4d00" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44397 } } }
I20260806 18:39:41.825661 10036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:41.825661 10035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:41.826224 10047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:41.829295 10047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:41.834542 10047 catalog_manager.cc:1347] Generated new cluster ID: 7dfb41c3061347999b8a5d762ea66ba1
I20260806 18:39:41.834620 10047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:41.845422 10047 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:41.846809 10047 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:41.855473 10047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b63bab267a04f9083b9ee2003ae4d00: Generated new TSK 0
I20260806 18:39:41.856388 10047 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:39:41.885918 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:44397
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:42.003568 10063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:42.003861 10063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:42.003927 10063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:42.007989 10063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:42.008057 10063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:42.008143 10063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:39:42.013549 10063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:44397
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:42.014645 10063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:42.015699 10063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:42.025364 10070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:42.025393 10073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:42.025358 10069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:42.025707 10063 server_base.cc:1034] running on GCE node
I20260806 18:39:42.026144 10063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:42.026852 10063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:42.028029 10063 hybrid_clock.cc:648] HybridClock initialized: now 1786041582027999 us; error 42 us; skew 500 ppm
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:42.030865 10063 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:39:42.031829 10063 webserver.cc:460] Webserver started at http://127.8.241.193:40233/ using document root <none> and password file <none>
I20260806 18:39:42.032407 10063 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:42.032478 10063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:42.032691 10063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:42.034878 10063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "9e31319ef36e4e4e8a7d46e6667b251a"
format_stamp: "Formatted at 2026-08-06 18:39:42 on dist-test-slave-k5rr"
server_key: "c6af296d7972b0485e71455074239f26"
server_key_iv: "b8afe898ffe295770c1dacaed4c4c196"
server_key_version: "encryptionkey@0"
I20260806 18:39:42.035492 10063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "9e31319ef36e4e4e8a7d46e6667b251a"
format_stamp: "Formatted at 2026-08-06 18:39:42 on dist-test-slave-k5rr"
server_key: "c6af296d7972b0485e71455074239f26"
server_key_iv: "b8afe898ffe295770c1dacaed4c4c196"
server_key_version: "encryptionkey@0"
I20260806 18:39:42.039543 10063 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:39:42.042115 10083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:42.043557 10063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:42.043689 10063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "9e31319ef36e4e4e8a7d46e6667b251a"
format_stamp: "Formatted at 2026-08-06 18:39:42 on dist-test-slave-k5rr"
server_key: "c6af296d7972b0485e71455074239f26"
server_key_iv: "b8afe898ffe295770c1dacaed4c4c196"
server_key_version: "encryptionkey@0"
I20260806 18:39:42.043814 10063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:42.057292 10063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:42.060294 10063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:42.060495 10063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:42.061167 10063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:42.062063 10063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:42.062115 10063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:42.062170 10063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:42.062197 10063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:42.071182 10063 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:35139
I20260806 18:39:42.071202 10249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:35139 every 8 connection(s)
I20260806 18:39:42.072165 10063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:39:42.073665 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10063
I20260806 18:39:42.073783 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:39:42.074077 9159 external_mini_cluster.cc:1428] Setting key ec85034753589a62745b6f7a5e09b50c
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:39:42.086063 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.074162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:50271 (local address 127.8.241.254:44397)
0806 18:39:42.074288 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.074292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.075296 (+ 1004us) server_negotiation.cc:407] Connection header received
0806 18:39:42.076638 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.076641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.076696 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.076770 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:42.078018 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.078559 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.079695 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.079973 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.080063 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.082651 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:42.082656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:42.082659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:42.082684 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:42.084069 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.084544 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.084547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.084549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.084626 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.084949 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.084952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.084954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.085167 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:42.085256 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.085746 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.085887 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:39:42.086900 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.074668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:44397 (local address 127.8.241.193:50271)
0806 18:39:42.075146 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:42.075189 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:42.076426 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:42.076966 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:42.076973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:42.077354 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:42.077866 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.077881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.078683 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.078686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.079570 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.079580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.080133 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.080138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.080297 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.080846 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:42.080863 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:42.082483 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:42.084178 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:42.084184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:42.084201 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:42.084450 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:42.084713 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:42.084716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:42.084718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:42.084860 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:42.085267 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:42.085273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:42.085516 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 18:39:42.085690 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 18:39:42.086023 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
I20260806 18:39:42.088111 10250 heartbeater.cc:347] Connected to a master server at 127.8.241.254:44397
I20260806 18:39:42.088366 10250 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:42.088860 10250 heartbeater.cc:511] Master 127.8.241.254:44397 requested a full tablet report, sending...
I20260806 18:39:42.090627 9975 ts_manager.cc:194] Registered new tserver with Master: 9e31319ef36e4e4e8a7d46e6667b251a (127.8.241.193:35139)
I20260806 18:39:42.092211 9975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:50271
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:39:42.102079 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:44397
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:42.217865 10256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:42.218148 10256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:42.218211 10256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:42.222225 10256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:42.222288 10256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:42.222371 10256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:39:42.228157 10256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:44397
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:42.229346 10256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:42.230396 10256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:42.240202 10267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:42.240231 10265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:42.240223 10264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:42.240315 10256 server_base.cc:1034] running on GCE node
I20260806 18:39:42.240917 10256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:42.241592 10256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:42.242759 10256 hybrid_clock.cc:648] HybridClock initialized: now 1786041582242736 us; error 32 us; skew 500 ppm
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:42.245515 10256 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:39:42.246394 10256 webserver.cc:460] Webserver started at http://127.8.241.194:37217/ using document root <none> and password file <none>
I20260806 18:39:42.246973 10256 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:42.247038 10256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:42.247218 10256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:42.249365 10256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "8a18c8d74c0e412d93604a792530cf5d"
format_stamp: "Formatted at 2026-08-06 18:39:42 on dist-test-slave-k5rr"
server_key: "0b8cc8870ee94767432beb30643015b1"
server_key_iv: "856ff93067115b47b5f95f3a5aee75c5"
server_key_version: "encryptionkey@0"
I20260806 18:39:42.249884 10256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "8a18c8d74c0e412d93604a792530cf5d"
format_stamp: "Formatted at 2026-08-06 18:39:42 on dist-test-slave-k5rr"
server_key: "0b8cc8870ee94767432beb30643015b1"
server_key_iv: "856ff93067115b47b5f95f3a5aee75c5"
server_key_version: "encryptionkey@0"
I20260806 18:39:42.253741 10256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:39:42.256254 10279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:42.257606 10256 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:39:42.257714 10256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "8a18c8d74c0e412d93604a792530cf5d"
format_stamp: "Formatted at 2026-08-06 18:39:42 on dist-test-slave-k5rr"
server_key: "0b8cc8870ee94767432beb30643015b1"
server_key_iv: "856ff93067115b47b5f95f3a5aee75c5"
server_key_version: "encryptionkey@0"
I20260806 18:39:42.257817 10256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:42.276232 10256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:42.279770 10256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:42.280032 10256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:42.280772 10256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:42.281875 10256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:42.281970 10256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:42.282042 10256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:42.282078 10256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:42.291925 10256 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:41281
I20260806 18:39:42.291949 10454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:41281 every 8 connection(s)
I20260806 18:39:42.292856 10256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:39:42.298929 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10256
I20260806 18:39:42.299057 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:39:42.299325 9159 external_mini_cluster.cc:1428] Setting key 21a6e2ad24c36d4d6901c11a4e1a3f9b
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:42.306821 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.294766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:59971 (local address 127.8.241.254:44397)
0806 18:39:42.294898 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.294901 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.295842 (+ 941us) server_negotiation.cc:407] Connection header received
0806 18:39:42.296938 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.296940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.296990 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.297092 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:42.298289 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.298901 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.300088 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.300351 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.300436 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.303236 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:42.303239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:42.303241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:42.303264 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:42.304588 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.305154 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.305158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.305161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.305264 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.305616 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.305620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.305622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.305833 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:42.305932 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.306478 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.306616 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":54}
I20260806 18:39:42.307667 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.295141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:44397 (local address 127.8.241.194:59971)
0806 18:39:42.295665 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:42.295731 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:42.296746 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:42.297253 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:42.297261 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:42.297656 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:42.298146 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.298161 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.299017 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.299020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.299955 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.299968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.300515 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.300519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.300719 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.301334 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:42.301350 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:42.303053 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:42.304702 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:42.304709 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:42.304726 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:42.305049 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:42.305361 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:42.305365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:42.305367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:42.305517 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:42.305945 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:42.305952 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:42.306225 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 18:39:42.306419 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 18:39:42.306774 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260806 18:39:42.308996 10455 heartbeater.cc:347] Connected to a master server at 127.8.241.254:44397
I20260806 18:39:42.309294 10455 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:42.309823 10455 heartbeater.cc:511] Master 127.8.241.254:44397 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:39:42.311028 9975 ts_manager.cc:194] Registered new tserver with Master: 8a18c8d74c0e412d93604a792530cf5d (127.8.241.194:41281)
I20260806 18:39:42.311625 9975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:59971
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:39:42.327942 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:44397
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:42.445287 10462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:42.445562 10462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:42.445624 10462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:42.449728 10462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:42.449801 10462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:42.449883 10462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:39:42.455160 10462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:44397
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:42.456257 10462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:42.457391 10462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:42.467696 10482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:42.467705 10477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:42.467705 10478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:42.467940 10462 server_base.cc:1034] running on GCE node
I20260806 18:39:42.468453 10462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:42.469151 10462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:42.470326 10462 hybrid_clock.cc:648] HybridClock initialized: now 1786041582470303 us; error 33 us; skew 500 ppm
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:42.473165 10462 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:39:42.474092 10462 webserver.cc:460] Webserver started at http://127.8.241.195:33817/ using document root <none> and password file <none>
I20260806 18:39:42.474659 10462 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:42.474725 10462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:42.474980 10462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:42.477231 10462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "2c605f8a8e454b9488780fcc6675bdde"
format_stamp: "Formatted at 2026-08-06 18:39:42 on dist-test-slave-k5rr"
server_key: "e438a63e35be528fe2aeab481b3e6170"
server_key_iv: "517de1e98492ef58b844ba4368f172c4"
server_key_version: "encryptionkey@0"
I20260806 18:39:42.477826 10462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "2c605f8a8e454b9488780fcc6675bdde"
format_stamp: "Formatted at 2026-08-06 18:39:42 on dist-test-slave-k5rr"
server_key: "e438a63e35be528fe2aeab481b3e6170"
server_key_iv: "517de1e98492ef58b844ba4368f172c4"
server_key_version: "encryptionkey@0"
I20260806 18:39:42.481732 10462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:39:42.484387 10495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:42.485754 10462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:42.485893 10462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "2c605f8a8e454b9488780fcc6675bdde"
format_stamp: "Formatted at 2026-08-06 18:39:42 on dist-test-slave-k5rr"
server_key: "e438a63e35be528fe2aeab481b3e6170"
server_key_iv: "517de1e98492ef58b844ba4368f172c4"
server_key_version: "encryptionkey@0"
I20260806 18:39:42.485989 10462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:42.506785 10462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:42.510066 10462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:42.510239 10462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:42.510965 10462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:42.511896 10462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:42.511947 10462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:42.511991 10462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:42.512012 10462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:42.520804 10462 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:45671
I20260806 18:39:42.520828 10654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:45671 every 8 connection(s)
I20260806 18:39:42.521766 10462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:39:42.524977 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10462
I20260806 18:39:42.525143 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:39:42.525468 9159 external_mini_cluster.cc:1428] Setting key ce128c141f9478a5c884816231144b5a
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:42.535673 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.523630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:41323 (local address 127.8.241.254:44397)
0806 18:39:42.523762 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.523765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.524687 (+ 922us) server_negotiation.cc:407] Connection header received
0806 18:39:42.526328 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.526331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.526387 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.526465 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:42.527675 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.528206 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.529390 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.529613 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.529703 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.532226 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:42.532230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:42.532232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:42.532257 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:42.533539 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.534038 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.534041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.534042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.534120 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.534434 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.534438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.534439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.534626 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:42.534719 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.535372 (+ 653us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.535504 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:39:42.536559 10660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.524070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:44397 (local address 127.8.241.195:41323)
0806 18:39:42.524538 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:42.524582 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:42.526117 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:42.526659 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:42.526667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:42.527024 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:42.527530 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.527545 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.528325 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.528328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.529271 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.529282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.529759 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.529764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.529927 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.530505 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:42.530522 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:42.531972 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:42.533654 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:42.533661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:42.533678 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:42.533939 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:42.534202 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:42.534205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:42.534207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:42.534345 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:42.534727 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:42.534734 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:42.535084 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 18:39:42.535293 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 18:39:42.535644 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260806 18:39:42.537716 10655 heartbeater.cc:347] Connected to a master server at 127.8.241.254:44397
I20260806 18:39:42.537981 10655 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:42.538519 10655 heartbeater.cc:511] Master 127.8.241.254:44397 requested a full tablet report, sending...
I20260806 18:39:42.539570 9975 ts_manager.cc:194] Registered new tserver with Master: 2c605f8a8e454b9488780fcc6675bdde (127.8.241.195:45671)
I20260806 18:39:42.540105 9975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:41323
I20260806 18:39:42.551299 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:39:42.562249 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.555147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.8.241.254:44397)
0806 18:39:42.555276 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.555280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.555452 (+ 172us) server_negotiation.cc:407] Connection header received
0806 18:39:42.555564 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.555567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.555611 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.555686 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:42.556382 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.556886 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.557727 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.557943 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.558031 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.559279 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:42.559282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:42.559284 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:42.559308 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:42.560469 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.560906 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.560909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.560911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.561033 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.561313 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.561316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.561318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.561552 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:42.561709 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.561940 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.562069 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:39:42.567866 9975 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:39:42.571058 9975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:39:42.586519 10679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.580637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:41281 (local address 127.0.0.1:36930)
0806 18:39:42.581545 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:42.581558 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:42.581652 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:42.582407 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:42.582419 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:42.582465 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:42.582607 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.582613 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.583680 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.583683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.584896 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.584908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.585949 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.585953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.586069 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.586108 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:39:42.586216 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 18:39:42.586350 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":2,"spinlock_wait_cycles":4736,"thread_start_us":150,"threads_started":1}
I20260806 18:39:42.586521 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.580830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36930 (local address 127.8.241.194:41281)
0806 18:39:42.581601 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.581613 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.581638 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:39:42.581814 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.581821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.582066 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.582299 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:42.582724 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.583555 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.585083 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.585833 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.585933 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.586100 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.586258 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.586359 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":173,"threads_started":1}
I20260806 18:39:42.586905 10677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.580515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:45671 (local address 127.0.0.1:37600)
0806 18:39:42.581417 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:42.581471 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:42.581610 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:42.582423 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:42.582427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:42.582461 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:42.582638 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.582645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.584125 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.584129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.585242 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.585251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.586475 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.586484 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.586611 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.586635 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:39:42.586703 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:39:42.586766 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":75,"spinlock_wait_cycles":4480,"thread_start_us":99,"threads_started":1}
I20260806 18:39:42.587365 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.580884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37600 (local address 127.8.241.195:45671)
0806 18:39:42.581391 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.581405 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.581759 (+ 354us) server_negotiation.cc:407] Connection header received
0806 18:39:42.581846 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.581851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.582067 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.582221 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:42.582772 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.583974 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.585348 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.586142 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.586251 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.586391 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.587106 (+ 715us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.587209 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":185,"threads_started":1}
I20260806 18:39:42.587378 10682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.581312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:35139 (local address 127.0.0.1:48990)
0806 18:39:42.581767 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:42.581779 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:42.581850 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:42.582760 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:42.582763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:42.582782 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:42.582857 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.582862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.584147 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.584149 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.585456 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.585469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.586907 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.586911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.587131 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.587154 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:39:42.587206 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 18:39:42.587267 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260806 18:39:42.588110 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.582041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48990 (local address 127.8.241.193:35139)
0806 18:39:42.582337 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.582353 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.582374 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:39:42.582455 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.582460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.582638 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.582792 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:42.582945 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.584010 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.585587 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.586408 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.586514 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.587578 (+ 1064us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.587635 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.587739 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260806 18:39:42.590044 10332 tablet_service.cc:1467] Processing CreateTablet for tablet d0da2ba5bcfe4854b6d594e4b216af78 (DEFAULT_TABLE table=test-table [id=f4b2fa7d6c4346afbd1e9b903f11e366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:42.590330 10562 tablet_service.cc:1467] Processing CreateTablet for tablet d0da2ba5bcfe4854b6d594e4b216af78 (DEFAULT_TABLE table=test-table [id=f4b2fa7d6c4346afbd1e9b903f11e366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:42.591250 10562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0da2ba5bcfe4854b6d594e4b216af78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:42.591261 10332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0da2ba5bcfe4854b6d594e4b216af78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:42.590955 10154 tablet_service.cc:1467] Processing CreateTablet for tablet d0da2ba5bcfe4854b6d594e4b216af78 (DEFAULT_TABLE table=test-table [id=f4b2fa7d6c4346afbd1e9b903f11e366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:42.591934 10154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0da2ba5bcfe4854b6d594e4b216af78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:42.597491 10684 tablet_bootstrap.cc:492] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde: Bootstrap starting.
I20260806 18:39:42.599035 10685 tablet_bootstrap.cc:492] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a: Bootstrap starting.
I20260806 18:39:42.600293 10684 tablet_bootstrap.cc:654] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:42.600723 10685 tablet_bootstrap.cc:654] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:42.601361 10686 tablet_bootstrap.cc:492] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d: Bootstrap starting.
I20260806 18:39:42.601397 10684 log.cc:824] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:42.601531 10685 log.cc:824] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:42.603341 10685 tablet_bootstrap.cc:492] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a: No bootstrap required, opened a new log
I20260806 18:39:42.603461 10684 tablet_bootstrap.cc:492] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde: No bootstrap required, opened a new log
I20260806 18:39:42.603554 10685 ts_tablet_manager.cc:1397] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:39:42.603657 10684 ts_tablet_manager.cc:1397] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:39:42.603935 10686 tablet_bootstrap.cc:654] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:42.604784 10686 log.cc:824] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:42.606956 10686 tablet_bootstrap.cc:492] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d: No bootstrap required, opened a new log
I20260806 18:39:42.607160 10686 ts_tablet_manager.cc:1397] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:39:42.607161 10684 raft_consensus.cc:348] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } }
I20260806 18:39:42.607430 10684 raft_consensus.cc:374] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:42.607486 10684 raft_consensus.cc:729] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c605f8a8e454b9488780fcc6675bdde, State: Initialized, Role: FOLLOWER
I20260806 18:39:42.607661 10685 raft_consensus.cc:348] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } }
I20260806 18:39:42.607929 10685 raft_consensus.cc:374] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:42.607990 10685 raft_consensus.cc:729] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e31319ef36e4e4e8a7d46e6667b251a, State: Initialized, Role: FOLLOWER
I20260806 18:39:42.607999 10684 consensus_queue.cc:260] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } }
I20260806 18:39:42.608433 10685 consensus_queue.cc:260] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } }
I20260806 18:39:42.608745 10655 heartbeater.cc:503] Master 127.8.241.254:44397 was elected leader, sending a full tablet report...
I20260806 18:39:42.609114 10684 ts_tablet_manager.cc:1428] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260806 18:39:42.609409 10685 ts_tablet_manager.cc:1428] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:39:42.609701 10250 heartbeater.cc:503] Master 127.8.241.254:44397 was elected leader, sending a full tablet report...
I20260806 18:39:42.610955 10686 raft_consensus.cc:348] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } }
I20260806 18:39:42.611231 10686 raft_consensus.cc:374] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:42.611292 10686 raft_consensus.cc:729] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a18c8d74c0e412d93604a792530cf5d, State: Initialized, Role: FOLLOWER
I20260806 18:39:42.611680 10686 consensus_queue.cc:260] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } }
I20260806 18:39:42.612270 10455 heartbeater.cc:503] Master 127.8.241.254:44397 was elected leader, sending a full tablet report...
I20260806 18:39:42.613410 10686 ts_tablet_manager.cc:1428] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260806 18:39:42.772794 10657 tablet.cc:2369] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:39:42.776647 10692 raft_consensus.cc:482] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:39:42.776898 10692 raft_consensus.cc:504] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } }
I20260806 18:39:42.778187 10692 leader_election.cc:288] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c605f8a8e454b9488780fcc6675bdde (127.8.241.195:45671), 8a18c8d74c0e412d93604a792530cf5d (127.8.241.194:41281)
I20260806 18:39:42.782616 10680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.778576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:33721 (local address 127.8.241.194:41281)
0806 18:39:42.778738 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.778743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.778979 (+ 236us) server_negotiation.cc:407] Connection header received
0806 18:39:42.779144 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.779148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.779197 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.779264 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:42.779604 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.780111 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.781348 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.781940 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.782061 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.782116 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.782389 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.782478 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:39:42.782641 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.778549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:48155 (local address 127.8.241.195:45671)
0806 18:39:42.778746 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.778750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.779112 (+ 362us) server_negotiation.cc:407] Connection header received
0806 18:39:42.779234 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.779236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.779277 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.779337 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:42.779633 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.780112 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.781322 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.781925 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.782062 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.782116 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.782426 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.782505 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 18:39:42.782655 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.778473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:45671 (local address 127.8.241.193:48155)
0806 18:39:42.779009 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:42.779020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:42.779130 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:42.779357 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:42.779359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:42.779375 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:42.779524 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.779530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.780213 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.780217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.781166 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.781178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.782192 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.782195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.782278 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.782297 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:39:42.782368 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:39:42.782489 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":89,"thread_start_us":124,"threads_started":1}
I20260806 18:39:42.782667 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.778467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:41281 (local address 127.8.241.193:33721)
0806 18:39:42.778881 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:42.778893 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:42.779033 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:42.779298 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:42.779301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:42.779319 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:42.779492 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.779499 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.780231 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.780235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.781203 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:42.781234 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.782161 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.782165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:42.782248 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.782277 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:39:42.782366 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 18:39:42.782505 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":108,"thread_start_us":138,"threads_started":1}
I20260806 18:39:42.783496 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0da2ba5bcfe4854b6d594e4b216af78" candidate_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c605f8a8e454b9488780fcc6675bdde" is_pre_election: true
I20260806 18:39:42.783490 10391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0da2ba5bcfe4854b6d594e4b216af78" candidate_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a18c8d74c0e412d93604a792530cf5d" is_pre_election: true
I20260806 18:39:42.783797 10593 raft_consensus.cc:2454] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e31319ef36e4e4e8a7d46e6667b251a in term 0.
I20260806 18:39:42.783816 10391 raft_consensus.cc:2454] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e31319ef36e4e4e8a7d46e6667b251a in term 0.
I20260806 18:39:42.784288 10088 leader_election.cc:302] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2c605f8a8e454b9488780fcc6675bdde, 8a18c8d74c0e412d93604a792530cf5d, 9e31319ef36e4e4e8a7d46e6667b251a; no voters:
I20260806 18:39:42.784566 10692 raft_consensus.cc:2781] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:39:42.784658 10692 raft_consensus.cc:482] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:39:42.784708 10692 raft_consensus.cc:3037] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:42.785804 10692 raft_consensus.cc:504] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } }
I20260806 18:39:42.786146 10692 leader_election.cc:288] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [CANDIDATE]: Term 1 election: Requested vote from peers 2c605f8a8e454b9488780fcc6675bdde (127.8.241.195:45671), 8a18c8d74c0e412d93604a792530cf5d (127.8.241.194:41281)
I20260806 18:39:42.786504 10593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0da2ba5bcfe4854b6d594e4b216af78" candidate_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c605f8a8e454b9488780fcc6675bdde"
I20260806 18:39:42.786625 10593 raft_consensus.cc:3037] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:42.787679 10593 raft_consensus.cc:2454] T d0da2ba5bcfe4854b6d594e4b216af78 P 2c605f8a8e454b9488780fcc6675bdde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e31319ef36e4e4e8a7d46e6667b251a in term 1.
I20260806 18:39:42.788002 10391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0da2ba5bcfe4854b6d594e4b216af78" candidate_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a18c8d74c0e412d93604a792530cf5d"
I20260806 18:39:42.788030 10088 leader_election.cc:302] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c605f8a8e454b9488780fcc6675bdde, 9e31319ef36e4e4e8a7d46e6667b251a; no voters:
I20260806 18:39:42.788120 10391 raft_consensus.cc:3037] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:42.788205 10692 raft_consensus.cc:2781] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:42.788429 10692 raft_consensus.cc:686] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [term 1 LEADER]: Becoming Leader. State: Replica: 9e31319ef36e4e4e8a7d46e6667b251a, State: Running, Role: LEADER
I20260806 18:39:42.788779 10692 consensus_queue.cc:237] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } }
I20260806 18:39:42.789342 10391 raft_consensus.cc:2454] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e31319ef36e4e4e8a7d46e6667b251a in term 1.
I20260806 18:39:42.792479 9975 catalog_manager.cc:5495] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e31319ef36e4e4e8a7d46e6667b251a (127.8.241.193). New cstate: current_term: 1 leader_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c605f8a8e454b9488780fcc6675bdde" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a18c8d74c0e412d93604a792530cf5d" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e31319ef36e4e4e8a7d46e6667b251a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35139 } health_report { overall_health: HEALTHY } } }
W20260806 18:39:42.794085 10457 tablet.cc:2369] T d0da2ba5bcfe4854b6d594e4b216af78 P 8a18c8d74c0e412d93604a792530cf5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
W20260806 18:39:42.823544 10251 tablet.cc:2369] T d0da2ba5bcfe4854b6d594e4b216af78 P 9e31319ef36e4e4e8a7d46e6667b251a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:39:42.825968 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.817266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48994 (local address 127.8.241.193:35139)
0806 18:39:42.817438 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.817444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.817569 (+ 125us) server_negotiation.cc:407] Connection header received
0806 18:39:42.817711 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.817714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.817763 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.817832 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:42.818545 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.819055 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.819878 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.820100 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.820192 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.822544 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:42.822558 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:42.822575 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:42.822599 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:42.824240 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.824650 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.824656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.824657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.824744 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.825024 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.825028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.825030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.825305 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:42.825534 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.825695 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.825814 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:39:42.857583 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.844604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.8.241.193:35139)
0806 18:39:42.844776 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.844781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.844876 (+ 95us) server_negotiation.cc:407] Connection header received
0806 18:39:42.845049 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.845052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.845106 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.845174 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:42.845794 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.846306 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.847176 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.847436 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.847527 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.854469 (+ 6942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:42.854472 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:42.854475 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:42.854507 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:42.855962 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.856392 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.856395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.856396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.856480 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.856672 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.856674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.856676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.856847 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:42.856976 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.857289 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.857402 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:39:42 dist-test-slave-k5rr krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:39:42.884004 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:42.875998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.8.241.193:35139)
0806 18:39:42.876133 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:42.876138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:42.876303 (+ 165us) server_negotiation.cc:407] Connection header received
0806 18:39:42.876473 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:42.876476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:42.876521 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:42.876593 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:42.877355 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.877861 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.878679 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:42.878898 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:42.878988 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:42.881345 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:42.881349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:42.881351 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:42.881377 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:42.882484 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.882868 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.882871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.882872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.882948 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:42.883191 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:42.883194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:42.883196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:42.883368 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:42.883505 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:42.883724 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 18:39:42.883849 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 18:39:42.885571 10119 tablet_service.cc:2908] Rejecting scan request for tablet d0da2ba5bcfe4854b6d594e4b216af78: Uninitialized: clean time has not yet been initialized
I20260806 18:39:42.887571 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10063
I20260806 18:39:42.911428 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10256
I20260806 18:39:42.921466 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10462
I20260806 18:39:42.933442 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9935
2026-08-06T18:39:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2607 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:43 dist-test-slave-k5rr krb5kdc[10720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:43 dist-test-slave-k5rr krb5kdc[10720](info): set up 2 sockets
Aug 06 18:39:43 dist-test-slave-k5rr krb5kdc[10720](info): commencing operation
krb5kdc: starting...
W20260806 18:39:44.115305 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:39:44 dist-test-slave-k5rr krb5kdc[10720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:39:44.244009 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45431
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:42949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:45431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:44.361275 10736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:44.361570 10736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:44.361626 10736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:44.365658 10736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:44.365729 10736 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:39:44.365754 10736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:44.365777 10736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:44.365804 10736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:44.365834 10736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:44.371676 10736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:45431
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:44.372951 10736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:44.374150 10736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:44.382886 10741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:44.382870 10742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:44.382867 10747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:44.383304 10736 server_base.cc:1034] running on GCE node
I20260806 18:39:44.383888 10736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:44.385090 10736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:44.386293 10736 hybrid_clock.cc:648] HybridClock initialized: now 1786041584386273 us; error 33 us; skew 500 ppm
Aug 06 18:39:44 dist-test-slave-k5rr krb5kdc[10720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041584, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:44.389398 10736 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:39:44.390482 10736 webserver.cc:460] Webserver started at http://127.8.241.254:46173/ using document root <none> and password file <none>
I20260806 18:39:44.391083 10736 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:44.391153 10736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:44.391352 10736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:44.393582 10736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "9d4b3cacea4e424aadebd520ea33f810"
format_stamp: "Formatted at 2026-08-06 18:39:44 on dist-test-slave-k5rr"
server_key: "de1b5b171fe369a15bdf4bcd131c6651"
server_key_iv: "f0c4447cc5970b70e92f7a721f39a9e4"
server_key_version: "encryptionkey@0"
I20260806 18:39:44.394153 10736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "9d4b3cacea4e424aadebd520ea33f810"
format_stamp: "Formatted at 2026-08-06 18:39:44 on dist-test-slave-k5rr"
server_key: "de1b5b171fe369a15bdf4bcd131c6651"
server_key_iv: "f0c4447cc5970b70e92f7a721f39a9e4"
server_key_version: "encryptionkey@0"
I20260806 18:39:44.398062 10736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:39:44.400585 10754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:44.401847 10736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:39:44.401970 10736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "9d4b3cacea4e424aadebd520ea33f810"
format_stamp: "Formatted at 2026-08-06 18:39:44 on dist-test-slave-k5rr"
server_key: "de1b5b171fe369a15bdf4bcd131c6651"
server_key_iv: "f0c4447cc5970b70e92f7a721f39a9e4"
server_key_version: "encryptionkey@0"
I20260806 18:39:44.402074 10736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:44.422091 10736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:44.425323 10736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:44.425515 10736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:44.433701 10736 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:45431
I20260806 18:39:44.433719 10843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:45431 every 8 connection(s)
I20260806 18:39:44.434731 10736 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:44.437789 10845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:44.441558 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10736
I20260806 18:39:44.441685 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:39:44.441962 9159 external_mini_cluster.cc:1428] Setting key f431713d35c9438b71f561e739364c7b
I20260806 18:39:44.444310 10845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810: Bootstrap starting.
I20260806 18:39:44.446753 10845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:44.447508 10845 log.cc:824] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:44.449502 10845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810: No bootstrap required, opened a new log
Aug 06 18:39:44 dist-test-slave-k5rr krb5kdc[10720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:44.453265 10845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4b3cacea4e424aadebd520ea33f810" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45431 } }
I20260806 18:39:44.453524 10845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:44.453584 10845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d4b3cacea4e424aadebd520ea33f810, State: Initialized, Role: FOLLOWER
I20260806 18:39:44.454128 10845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4b3cacea4e424aadebd520ea33f810" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45431 } }
I20260806 18:39:44.454304 10845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:44.454382 10845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:44.454489 10845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:44.455415 10845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4b3cacea4e424aadebd520ea33f810" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45431 } }
I20260806 18:39:44.455766 10845 leader_election.cc:302] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d4b3cacea4e424aadebd520ea33f810; no voters:
I20260806 18:39:44.456108 10845 leader_election.cc:288] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:44.456151 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:44.443485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39228 (local address 127.8.241.254:45431)
0806 18:39:44.443969 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:44.443986 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:44.444025 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:39:44.444684 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:44.444704 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:44.445118 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:44.445448 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:44.446260 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.447325 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.448200 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.448678 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.448829 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:44.451196 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:44.451202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:44.451217 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:44.451245 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:44.452840 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.453302 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.453307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.453313 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.453427 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.453709 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.453713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.453715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.454253 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:44.454486 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:44.455053 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 18:39:44.455311 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":146,"threads_started":1}
I20260806 18:39:44.456199 10853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:44.457374 10853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [term 1 LEADER]: Becoming Leader. State: Replica: 9d4b3cacea4e424aadebd520ea33f810, State: Running, Role: LEADER
I20260806 18:39:44.457504 10845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:44.457721 10853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4b3cacea4e424aadebd520ea33f810" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45431 } }
I20260806 18:39:44.460029 10856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d4b3cacea4e424aadebd520ea33f810. Latest consensus state: current_term: 1 leader_uuid: "9d4b3cacea4e424aadebd520ea33f810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4b3cacea4e424aadebd520ea33f810" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45431 } } }
I20260806 18:39:44.460146 10856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:44.459754 10855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d4b3cacea4e424aadebd520ea33f810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d4b3cacea4e424aadebd520ea33f810" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45431 } } }
I20260806 18:39:44.460220 10855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810 [sys.catalog]: This master's current role is: LEADER
W20260806 18:39:44.462556 10872 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:44.462627 10872 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:44.462707 10873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:44.463409 10873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:44.470057 10873 catalog_manager.cc:1347] Generated new cluster ID: 28b9f04f45fb4f0cb9ae6bd9a7837f51
I20260806 18:39:44.470153 10873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:44.479024 10873 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:44.480033 10873 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:44.485816 10873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810: Generated new TSK 0
I20260806 18:39:44.486716 10873 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:39:44.520758 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45431
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:42949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:44.667830 10882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:44.668125 10882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:44.668179 10882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:44.672320 10882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:44.672407 10882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:44.672447 10882 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:39:44.672483 10882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:39:44.678344 10882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.241.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:44.679514 10882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:44.680596 10882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:44.690901 10893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:44.690886 10892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:44.690886 10895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:44.691027 10882 server_base.cc:1034] running on GCE node
I20260806 18:39:44.691660 10882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:44.692399 10882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:44.693572 10882 hybrid_clock.cc:648] HybridClock initialized: now 1786041584693549 us; error 35 us; skew 500 ppm
Aug 06 18:39:44 dist-test-slave-k5rr krb5kdc[10720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041584, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:44.696447 10882 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:39:44.697474 10882 webserver.cc:460] Webserver started at http://127.8.241.193:44851/ using document root <none> and password file <none>
I20260806 18:39:44.698084 10882 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:44.698149 10882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:44.698386 10882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:44.700619 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "59329d7a2ed745dbb0cd77c85ae80522"
format_stamp: "Formatted at 2026-08-06 18:39:44 on dist-test-slave-k5rr"
server_key: "4822f4903a4afe7846b0a32feb783ffb"
server_key_iv: "e29c58c13b96ab2ec999e82a09e14fab"
server_key_version: "encryptionkey@0"
I20260806 18:39:44.701222 10882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "59329d7a2ed745dbb0cd77c85ae80522"
format_stamp: "Formatted at 2026-08-06 18:39:44 on dist-test-slave-k5rr"
server_key: "4822f4903a4afe7846b0a32feb783ffb"
server_key_iv: "e29c58c13b96ab2ec999e82a09e14fab"
server_key_version: "encryptionkey@0"
I20260806 18:39:44.705300 10882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:39:44.708048 10902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:44.709483 10882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:44.709615 10882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "59329d7a2ed745dbb0cd77c85ae80522"
format_stamp: "Formatted at 2026-08-06 18:39:44 on dist-test-slave-k5rr"
server_key: "4822f4903a4afe7846b0a32feb783ffb"
server_key_iv: "e29c58c13b96ab2ec999e82a09e14fab"
server_key_version: "encryptionkey@0"
I20260806 18:39:44.709739 10882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:44.740748 10882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:44.743955 10882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:44.744165 10882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:44.745885 10882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:44.745944 10882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:44.746004 10882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:44.746031 10882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:39:44 dist-test-slave-k5rr krb5kdc[10720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041584, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:44.757203 10882 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:41493
I20260806 18:39:44.757259 11075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:41493 every 8 connection(s)
I20260806 18:39:44.758441 10882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:39:44.759443 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10882
I20260806 18:39:44.759563 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:39:44.759879 9159 external_mini_cluster.cc:1428] Setting key 6208deba1060d4526c9a8905c15215d1
I20260806 18:39:44.760545 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:44.746709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:45723 (local address 127.8.241.254:45431)
0806 18:39:44.746831 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:44.746834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:44.748023 (+ 1189us) server_negotiation.cc:407] Connection header received
0806 18:39:44.749386 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:44.749390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:44.749453 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:44.749539 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:44.750824 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.751377 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.752553 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.752865 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.752982 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:44.756264 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:44.756268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:44.756270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:44.756300 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:44.758097 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.758647 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.758651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.758653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.758783 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.759208 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.759211 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.759213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.759446 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:44.759555 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:44.760232 (+ 677us) server_negotiation.cc:299] Negotiation successful
0806 18:39:44.760376 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:39:44.761453 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:44.747324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45431 (local address 127.8.241.193:45723)
0806 18:39:44.747858 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:44.747906 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:44.749139 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:44.749739 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:44.749747 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:44.750103 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:44.750666 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:44.750685 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.751520 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.751523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:44.752419 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:44.752430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.753057 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.753063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:44.753236 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:44.754043 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:44.754068 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:44.756055 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:44.758219 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:44.758225 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:44.758240 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:44.758537 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:44.758896 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:44.758899 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:44.758902 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:44.759108 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:44.759567 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:44.759575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:44.759937 (+ 362us) client_negotiation.cc:769] Sending connection context
0806 18:39:44.760185 (+ 248us) client_negotiation.cc:240] Negotiation successful
0806 18:39:44.760543 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":148,"threads_started":1}
I20260806 18:39:44.762794 11076 heartbeater.cc:347] Connected to a master server at 127.8.241.254:45431
I20260806 18:39:44.762965 11076 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:44.763600 11076 heartbeater.cc:511] Master 127.8.241.254:45431 requested a full tablet report, sending...
I20260806 18:39:44.765407 10779 ts_manager.cc:194] Registered new tserver with Master: 59329d7a2ed745dbb0cd77c85ae80522 (127.8.241.193:41493)
I20260806 18:39:44.766577 10779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:45723
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:39:44.775965 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:44.764982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45431 (local address 127.8.241.193:40981)
0806 18:39:44.769201 (+ 4219us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:44.769213 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:44.769318 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:44.769554 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:44.769557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:44.769760 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:44.769868 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:44.769875 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.770619 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.770623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:44.771209 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:44.771220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.771697 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.771700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:44.771779 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:44.772228 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:44.772239 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:44.772589 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:44.774461 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:44.774464 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:44.774467 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:44.774669 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:44.774977 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:44.774981 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:44.774983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:44.775098 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:44.775506 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:44.775508 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:44.775595 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 18:39:44.775670 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:39:44.775797 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4133,"thread_start_us":105,"threads_started":1}
I20260806 18:39:44.776016 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:44.765069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:40981 (local address 127.8.241.254:45431)
0806 18:39:44.765180 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:44.765184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:44.769288 (+ 4104us) server_negotiation.cc:407] Connection header received
0806 18:39:44.769404 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:44.769407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:44.769458 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:44.769523 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:44.769984 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.770496 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.771323 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.771565 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.771665 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:44.772733 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:44.772736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:44.772738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:44.772762 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:44.774340 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.774760 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.774763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.774765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.774848 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.775183 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.775186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.775187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.775424 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:44.775519 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:44.775738 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 18:39:44.775859 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:39:44.789939 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45431
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:42949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:44.908569 11087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:44.908849 11087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:44.908912 11087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:44.912930 11087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:44.913035 11087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:44.913095 11087 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:39:44.913148 11087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:39:44.918572 11087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.241.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:44.919719 11087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:44.920800 11087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:44.930824 11095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:44.930850 11092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:44.930824 11093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:44.931160 11087 server_base.cc:1034] running on GCE node
I20260806 18:39:44.931608 11087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:44.932264 11087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:44.933419 11087 hybrid_clock.cc:648] HybridClock initialized: now 1786041584933398 us; error 35 us; skew 500 ppm
Aug 06 18:39:44 dist-test-slave-k5rr krb5kdc[10720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041584, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:44.936101 11087 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:39:44.937076 11087 webserver.cc:460] Webserver started at http://127.8.241.194:44913/ using document root <none> and password file <none>
I20260806 18:39:44.937657 11087 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:44.937726 11087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:44.937927 11087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:44.940078 11087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "e37c54693a2a4558bda143ff2ea5ef71"
format_stamp: "Formatted at 2026-08-06 18:39:44 on dist-test-slave-k5rr"
server_key: "e2026b5f11827a87698696b7730b1798"
server_key_iv: "fdb455d1f7f22a8a3e1ab3c335e35181"
server_key_version: "encryptionkey@0"
I20260806 18:39:44.940678 11087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "e37c54693a2a4558bda143ff2ea5ef71"
format_stamp: "Formatted at 2026-08-06 18:39:44 on dist-test-slave-k5rr"
server_key: "e2026b5f11827a87698696b7730b1798"
server_key_iv: "fdb455d1f7f22a8a3e1ab3c335e35181"
server_key_version: "encryptionkey@0"
I20260806 18:39:44.944725 11087 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260806 18:39:44.947427 11106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:44.948702 11087 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:39:44.948829 11087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "e37c54693a2a4558bda143ff2ea5ef71"
format_stamp: "Formatted at 2026-08-06 18:39:44 on dist-test-slave-k5rr"
server_key: "e2026b5f11827a87698696b7730b1798"
server_key_iv: "fdb455d1f7f22a8a3e1ab3c335e35181"
server_key_version: "encryptionkey@0"
I20260806 18:39:44.948939 11087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:44.963335 11087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:44.966636 11087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:44.966888 11087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:44.968683 11087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:44.968735 11087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:44.968791 11087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:44.968814 11087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:39:44 dist-test-slave-k5rr krb5kdc[10720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041584, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:44.979518 11087 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:40115
I20260806 18:39:44.979964 11281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:40115 every 8 connection(s)
I20260806 18:39:44.980680 11087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:39:44.982765 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:44.969470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:57967 (local address 127.8.241.254:45431)
0806 18:39:44.969583 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:44.969587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:44.970673 (+ 1086us) server_negotiation.cc:407] Connection header received
0806 18:39:44.971692 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:44.971696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:44.971757 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:44.971814 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:44.973534 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.974139 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.975288 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.975612 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.975737 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:44.978862 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:44.978875 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:44.978878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:44.978909 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:44.980531 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.981099 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.981103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.981105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.981208 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.981616 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.981620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.981622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.981822 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:44.981912 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:44.982460 (+ 548us) server_negotiation.cc:299] Negotiation successful
0806 18:39:44.982595 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:39:44.983566 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:44.970021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45431 (local address 127.8.241.194:57967)
0806 18:39:44.970526 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:44.970570 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:44.971412 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:44.972258 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:44.972267 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:44.972745 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:44.973379 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:44.973397 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.974269 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.974272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:44.975165 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:44.975175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.975797 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.975803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:44.976040 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:44.976833 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:44.976856 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:44.978667 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:44.980646 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:44.980651 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:44.980665 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:44.980978 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:44.981318 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:44.981321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:44.981324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:44.981507 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:44.981930 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:44.981937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:44.982198 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 18:39:44.982369 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 18:39:44.982712 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":141,"threads_started":1}
I20260806 18:39:44.984723 11286 heartbeater.cc:347] Connected to a master server at 127.8.241.254:45431
I20260806 18:39:44.984877 11286 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:44.985442 11286 heartbeater.cc:511] Master 127.8.241.254:45431 requested a full tablet report, sending...
I20260806 18:39:44.986546 10780 ts_manager.cc:194] Registered new tserver with Master: e37c54693a2a4558bda143ff2ea5ef71 (127.8.241.194:40115)
I20260806 18:39:44.987242 10780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:57967
I20260806 18:39:44.987439 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 11087
I20260806 18:39:44.987545 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:39:44.987793 9159 external_mini_cluster.cc:1428] Setting key c82841753ba850ad43acbc9d59213db2
I20260806 18:39:44.994863 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:44.986925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45431 (local address 127.8.241.194:37967)
0806 18:39:44.987312 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:44.987327 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:44.987586 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:44.987942 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:44.987944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:44.988114 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:44.988280 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:44.988288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.989317 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.989320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:44.989975 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:44.989985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.990410 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.990414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:44.990510 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:44.990998 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:44.991010 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:44.991369 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:44.993239 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:44.993242 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:44.993243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:44.993469 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:44.993790 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:44.993793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:44.993794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:44.993876 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:44.994382 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:44.994385 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:44.994482 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 18:39:44.994574 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:39:44.994695 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260806 18:39:44.994966 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:44.987011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:37967 (local address 127.8.241.254:45431)
0806 18:39:44.987126 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:44.987131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:44.987391 (+ 260us) server_negotiation.cc:407] Connection header received
0806 18:39:44.987705 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:44.987708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:44.987757 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:44.987806 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:44.988412 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.989160 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.990091 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:44.990310 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:44.990395 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:44.991504 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:44.991508 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:44.991511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:44.991536 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:44.993092 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.993576 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.993580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.993582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.993689 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:44.993980 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:44.993983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:44.993985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:44.994291 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:44.994398 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:44.994654 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 18:39:44.994787 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:39:45.016187 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45431
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:42949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:45.135919 11298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:45.136240 11298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:45.136289 11298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:45.140504 11298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:45.140587 11298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:45.140630 11298 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:39:45.140662 11298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:39:45.146394 11298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.241.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:45.147547 11298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:45.148591 11298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:45.158753 11306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:45.158897 11298 server_base.cc:1034] running on GCE node
W20260806 18:39:45.158761 11307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:45.158771 11310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:45.159476 11298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:45.160178 11298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:45.161350 11298 hybrid_clock.cc:648] HybridClock initialized: now 1786041585161313 us; error 46 us; skew 500 ppm
Aug 06 18:39:45 dist-test-slave-k5rr krb5kdc[10720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041585, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:45.164368 11298 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:39:45.165377 11298 webserver.cc:460] Webserver started at http://127.8.241.195:36291/ using document root <none> and password file <none>
I20260806 18:39:45.165964 11298 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:45.166029 11298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:45.166244 11298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:45.168432 11298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "5cb45b01783f46a3b8db4e29cb5a75c2"
format_stamp: "Formatted at 2026-08-06 18:39:45 on dist-test-slave-k5rr"
server_key: "89b4eedec64d43cb0352deb2ee8cf500"
server_key_iv: "0492e2a1b52d2ca0354e15be91f60f6d"
server_key_version: "encryptionkey@0"
I20260806 18:39:45.169076 11298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "5cb45b01783f46a3b8db4e29cb5a75c2"
format_stamp: "Formatted at 2026-08-06 18:39:45 on dist-test-slave-k5rr"
server_key: "89b4eedec64d43cb0352deb2ee8cf500"
server_key_iv: "0492e2a1b52d2ca0354e15be91f60f6d"
server_key_version: "encryptionkey@0"
I20260806 18:39:45.173487 11298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260806 18:39:45.176318 11321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:45.177825 11298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:45.177960 11298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "5cb45b01783f46a3b8db4e29cb5a75c2"
format_stamp: "Formatted at 2026-08-06 18:39:45 on dist-test-slave-k5rr"
server_key: "89b4eedec64d43cb0352deb2ee8cf500"
server_key_iv: "0492e2a1b52d2ca0354e15be91f60f6d"
server_key_version: "encryptionkey@0"
I20260806 18:39:45.178097 11298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:45.189726 11298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:45.193226 11298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:45.193403 11298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:45.194995 11298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:45.195055 11298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:45.195113 11298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:45.195142 11298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:45.205667 11298 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:34259
Aug 06 18:39:45 dist-test-slave-k5rr krb5kdc[10720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041585, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:45.205761 11491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:34259 every 8 connection(s)
I20260806 18:39:45.206881 11298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:39:45.210500 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.195919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:56035 (local address 127.8.241.254:45431)
0806 18:39:45.196038 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.196041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.197332 (+ 1291us) server_negotiation.cc:407] Connection header received
0806 18:39:45.198414 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.198418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.198481 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.198574 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:45.200316 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.201186 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.202908 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.203126 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.203210 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.206443 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:45.206447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:45.206450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:45.206480 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:45.208200 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:45.208810 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:45.208814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:45.208815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:45.208918 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:45.209337 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:45.209340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:45.209341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:45.209543 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:45.209622 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.210193 (+ 571us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.210331 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:39:45.211402 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.196530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45431 (local address 127.8.241.195:56035)
0806 18:39:45.197167 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.197216 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.198165 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.198846 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.198858 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.199361 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:45.200119 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.200143 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.201332 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.201336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.202770 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.202781 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.203291 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.203295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.203524 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.204278 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:45.204303 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:45.206247 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:45.208345 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:45.208353 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:45.208372 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:45.208683 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:45.209066 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:45.209070 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:45.209072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:45.209244 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:45.209640 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:45.209646 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:45.209964 (+ 318us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.210135 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.210426 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":148,"threads_started":1}
I20260806 18:39:45.212803 11495 heartbeater.cc:347] Connected to a master server at 127.8.241.254:45431
I20260806 18:39:45.212973 11495 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:45.213637 11495 heartbeater.cc:511] Master 127.8.241.254:45431 requested a full tablet report, sending...
I20260806 18:39:45.214818 10780 ts_manager.cc:194] Registered new tserver with Master: 5cb45b01783f46a3b8db4e29cb5a75c2 (127.8.241.195:34259)
I20260806 18:39:45.215417 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 11298
I20260806 18:39:45.215543 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:39:45.215595 10780 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:56035
I20260806 18:39:45.215837 9159 external_mini_cluster.cc:1428] Setting key a39ec4f4ec6769e12978f498c4a6df2a
I20260806 18:39:45.217947 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:39:45.223727 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.215481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:43761 (local address 127.8.241.254:45431)
0806 18:39:45.215637 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.215642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.216369 (+ 727us) server_negotiation.cc:407] Connection header received
0806 18:39:45.216571 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.216574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.216624 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.216707 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:45.217465 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.218130 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.219181 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.219382 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.219459 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.220621 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:45.220625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:45.220628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:45.220656 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:45.222152 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:45.222593 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:45.222595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:45.222596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:45.222664 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:45.222907 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:45.222909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:45.222910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:45.223103 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:45.223176 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.223468 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.223566 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 18:39:45.223793 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.215392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45431 (local address 127.8.241.195:43761)
0806 18:39:45.216277 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.216292 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.216408 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.216932 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.216936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.217205 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:45.217329 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.217337 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.218272 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.218275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.219078 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.219091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.219476 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.219480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.219566 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.220091 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:45.220106 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:45.220475 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:45.222263 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:45.222267 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:45.222269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:45.222510 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:45.222734 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:45.222737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:45.222739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:45.222832 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:45.223182 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:45.223185 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:45.223300 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.223422 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.223583 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":144,"threads_started":1}
Aug 06 18:39:45 dist-test-slave-k5rr krb5kdc[10720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041585, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:39:45 dist-test-slave-k5rr krb5kdc[10720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041585, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:45.246445 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.238162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39238 (local address 127.8.241.254:45431)
0806 18:39:45.238308 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.238311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.238326 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:39:45.238370 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.238373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.238413 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.238485 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:45.239249 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.239775 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.240616 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.240830 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.240922 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.243387 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:45.243390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:45.243392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:45.243422 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:45.244740 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:45.245236 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:45.245239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:45.245240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:45.245322 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:45.245586 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:45.245590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:45.245592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:45.245782 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:45.245909 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.246161 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.246282 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:39:45.249593 10780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:39:45.252782 10780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:39:45.268237 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.262009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:41493 (local address 127.0.0.1:57600)
0806 18:39:45.263146 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.263204 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.263374 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.264086 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.264093 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.264116 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:45.264287 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.264295 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.265657 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.265660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.266830 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.266840 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.267660 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.267663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.267784 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.267825 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.267942 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.268079 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"thread_start_us":97,"threads_started":1}
I20260806 18:39:45.268249 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.262194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57600 (local address 127.8.241.193:41493)
0806 18:39:45.262646 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.262654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.263332 (+ 678us) server_negotiation.cc:407] Connection header received
0806 18:39:45.263639 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.263646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.263894 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.264443 (+ 549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:45.264592 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.265507 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.266974 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.267536 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.267637 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.267805 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.267995 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.268093 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":105,"threads_started":1}
I20260806 18:39:45.269343 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.262835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.8.241.195:34259)
0806 18:39:45.263630 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.263644 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.263968 (+ 324us) server_negotiation.cc:407] Connection header received
0806 18:39:45.264187 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.264194 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.264362 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.264507 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:45.264742 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.265939 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.267749 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.268574 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.268665 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.268886 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.269093 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.269198 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":127,"threads_started":1}
I20260806 18:39:45.269835 11528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.262684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:34259 (local address 127.0.0.1:57426)
0806 18:39:45.263869 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.263884 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.263998 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.264491 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.264494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.264514 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:45.264617 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.264625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.266101 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.266105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.267580 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.267593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.268804 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.268808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.268895 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.268943 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.269620 (+ 677us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.269703 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":86,"thread_start_us":109,"threads_started":1}
I20260806 18:39:45.269846 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.263285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60508 (local address 127.8.241.194:40115)
0806 18:39:45.263884 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.263898 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.264151 (+ 253us) server_negotiation.cc:407] Connection header received
0806 18:39:45.264232 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.264238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.264441 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.264592 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:45.264956 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.266097 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.267738 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.268822 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.268938 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.269141 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.269583 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.269688 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":156,"threads_started":1}
I20260806 18:39:45.270242 11522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.262933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:40115 (local address 127.0.0.1:60508)
0806 18:39:45.263937 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.263951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.264058 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.264729 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.264732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.264749 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:45.264830 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.264836 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.266238 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.266242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.267580 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.267593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.269225 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.269230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.269322 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.269348 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.269423 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.270086 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":781,"thread_start_us":109,"threads_started":1}
I20260806 18:39:45.272084 10976 tablet_service.cc:1467] Processing CreateTablet for tablet 95afcbfd37824c778fbf645397d1d0ef (DEFAULT_TABLE table=test-table [id=11000151ecf242b29c872148466da040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:45.272558 11395 tablet_service.cc:1467] Processing CreateTablet for tablet 95afcbfd37824c778fbf645397d1d0ef (DEFAULT_TABLE table=test-table [id=11000151ecf242b29c872148466da040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:45.273415 10976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95afcbfd37824c778fbf645397d1d0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:45.272974 11186 tablet_service.cc:1467] Processing CreateTablet for tablet 95afcbfd37824c778fbf645397d1d0ef (DEFAULT_TABLE table=test-table [id=11000151ecf242b29c872148466da040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:45.273612 11395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95afcbfd37824c778fbf645397d1d0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:45.274029 11186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95afcbfd37824c778fbf645397d1d0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:45.281445 11529 tablet_bootstrap.cc:492] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: Bootstrap starting.
I20260806 18:39:45.282281 11532 tablet_bootstrap.cc:492] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: Bootstrap starting.
I20260806 18:39:45.282713 11530 tablet_bootstrap.cc:492] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: Bootstrap starting.
I20260806 18:39:45.284410 11532 tablet_bootstrap.cc:654] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:45.284987 11530 tablet_bootstrap.cc:654] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:45.284988 11529 tablet_bootstrap.cc:654] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:45.285234 11532 log.cc:824] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:45.285890 11530 log.cc:824] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:45.286023 11529 log.cc:824] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:45.287331 11532 tablet_bootstrap.cc:492] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: No bootstrap required, opened a new log
I20260806 18:39:45.287545 11532 ts_tablet_manager.cc:1397] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:39:45.287948 11530 tablet_bootstrap.cc:492] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: No bootstrap required, opened a new log
I20260806 18:39:45.287945 11529 tablet_bootstrap.cc:492] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: No bootstrap required, opened a new log
I20260806 18:39:45.288144 11529 ts_tablet_manager.cc:1397] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260806 18:39:45.288144 11530 ts_tablet_manager.cc:1397] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260806 18:39:45.291136 11532 raft_consensus.cc:348] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.291401 11532 raft_consensus.cc:374] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:45.291476 11532 raft_consensus.cc:729] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e37c54693a2a4558bda143ff2ea5ef71, State: Initialized, Role: FOLLOWER
I20260806 18:39:45.291970 11532 consensus_queue.cc:260] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.292207 11530 raft_consensus.cc:348] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.292503 11530 raft_consensus.cc:374] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:45.292568 11530 raft_consensus.cc:729] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb45b01783f46a3b8db4e29cb5a75c2, State: Initialized, Role: FOLLOWER
I20260806 18:39:45.292873 11286 heartbeater.cc:503] Master 127.8.241.254:45431 was elected leader, sending a full tablet report...
I20260806 18:39:45.293136 11530 consensus_queue.cc:260] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.293330 11529 raft_consensus.cc:348] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.293613 11529 raft_consensus.cc:374] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:45.293673 11529 raft_consensus.cc:729] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59329d7a2ed745dbb0cd77c85ae80522, State: Initialized, Role: FOLLOWER
I20260806 18:39:45.294034 11495 heartbeater.cc:503] Master 127.8.241.254:45431 was elected leader, sending a full tablet report...
I20260806 18:39:45.294130 11529 consensus_queue.cc:260] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.294353 11530 ts_tablet_manager.cc:1428] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260806 18:39:45.295004 11076 heartbeater.cc:503] Master 127.8.241.254:45431 was elected leader, sending a full tablet report...
I20260806 18:39:45.295102 11529 ts_tablet_manager.cc:1428] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260806 18:39:45.295761 11532 ts_tablet_manager.cc:1428] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 18:39:45.345885 11540 raft_consensus.cc:482] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:39:45.346144 11540 raft_consensus.cc:504] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.347440 11540 leader_election.cc:288] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e37c54693a2a4558bda143ff2ea5ef71 (127.8.241.194:40115), 5cb45b01783f46a3b8db4e29cb5a75c2 (127.8.241.195:34259)
I20260806 18:39:45.352820 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.347834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:49877 (local address 127.8.241.194:40115)
0806 18:39:45.348007 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.348012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.348154 (+ 142us) server_negotiation.cc:407] Connection header received
0806 18:39:45.348305 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.348308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.348364 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.348454 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:45.348856 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.349692 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.351547 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.352155 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.352282 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.352343 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.352577 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.352653 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 18:39:45.352802 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.347832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:41599 (local address 127.8.241.195:34259)
0806 18:39:45.348108 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.348112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.348241 (+ 129us) server_negotiation.cc:407] Connection header received
0806 18:39:45.348343 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.348345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.348389 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.348457 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:45.348838 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.349693 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.351547 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.352155 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.352303 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.352347 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.352577 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.352653 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260806 18:39:45.352890 11541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.347716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:34259 (local address 127.8.241.193:41599)
0806 18:39:45.348164 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.348175 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.348259 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.348478 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.348481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.348518 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:45.348718 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.348727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.349799 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.349803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.351385 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.351397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.352275 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.352279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.352437 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.352468 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.352550 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.352683 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":214,"spinlock_wait_cycles":20736,"thread_start_us":151,"threads_started":1}
I20260806 18:39:45.353245 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.347699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:40115 (local address 127.8.241.193:49877)
0806 18:39:45.348051 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.348067 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.348203 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.348478 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.348481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.348521 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:45.348719 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.348727 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.349799 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.349803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.351383 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.351397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.352275 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.352279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.352441 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.352472 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.352550 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.352683 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":252,"spinlock_wait_cycles":7296,"thread_start_us":166,"threads_started":1}
I20260806 18:39:45.353677 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95afcbfd37824c778fbf645397d1d0ef" candidate_uuid: "59329d7a2ed745dbb0cd77c85ae80522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" is_pre_election: true
I20260806 18:39:45.353838 11211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95afcbfd37824c778fbf645397d1d0ef" candidate_uuid: "59329d7a2ed745dbb0cd77c85ae80522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e37c54693a2a4558bda143ff2ea5ef71" is_pre_election: true
I20260806 18:39:45.354007 11426 raft_consensus.cc:2454] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59329d7a2ed745dbb0cd77c85ae80522 in term 0.
I20260806 18:39:45.354135 11211 raft_consensus.cc:2454] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59329d7a2ed745dbb0cd77c85ae80522 in term 0.
I20260806 18:39:45.354494 10904 leader_election.cc:302] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59329d7a2ed745dbb0cd77c85ae80522, 5cb45b01783f46a3b8db4e29cb5a75c2; no voters:
I20260806 18:39:45.354775 11540 raft_consensus.cc:2781] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:39:45.354879 11540 raft_consensus.cc:482] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:39:45.354918 11540 raft_consensus.cc:3037] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:45.356128 11540 raft_consensus.cc:504] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.356506 11540 leader_election.cc:288] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [CANDIDATE]: Term 1 election: Requested vote from peers e37c54693a2a4558bda143ff2ea5ef71 (127.8.241.194:40115), 5cb45b01783f46a3b8db4e29cb5a75c2 (127.8.241.195:34259)
I20260806 18:39:45.356896 11211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95afcbfd37824c778fbf645397d1d0ef" candidate_uuid: "59329d7a2ed745dbb0cd77c85ae80522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e37c54693a2a4558bda143ff2ea5ef71"
I20260806 18:39:45.357061 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95afcbfd37824c778fbf645397d1d0ef" candidate_uuid: "59329d7a2ed745dbb0cd77c85ae80522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2"
I20260806 18:39:45.357079 11211 raft_consensus.cc:3037] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:45.357196 11426 raft_consensus.cc:3037] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:45.358498 11211 raft_consensus.cc:2454] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59329d7a2ed745dbb0cd77c85ae80522 in term 1.
I20260806 18:39:45.358498 11426 raft_consensus.cc:2454] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59329d7a2ed745dbb0cd77c85ae80522 in term 1.
I20260806 18:39:45.358855 10904 leader_election.cc:302] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 59329d7a2ed745dbb0cd77c85ae80522, 5cb45b01783f46a3b8db4e29cb5a75c2, e37c54693a2a4558bda143ff2ea5ef71; no voters:
I20260806 18:39:45.359081 11540 raft_consensus.cc:2781] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:45.359347 11540 raft_consensus.cc:686] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 LEADER]: Becoming Leader. State: Replica: 59329d7a2ed745dbb0cd77c85ae80522, State: Running, Role: LEADER
I20260806 18:39:45.359754 11540 consensus_queue.cc:237] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.363247 10779 catalog_manager.cc:5495] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59329d7a2ed745dbb0cd77c85ae80522 (127.8.241.193). New cstate: current_term: 1 leader_uuid: "59329d7a2ed745dbb0cd77c85ae80522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260806 18:39:45.402833 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.399243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57604 (local address 127.8.241.193:41493)
0806 18:39:45.399414 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.399419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.399437 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:39:45.399624 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.399626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.399686 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.399771 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:45.399999 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.400505 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.401412 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.401643 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.401726 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.402000 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:45.402463 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:45.402549 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.402623 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.402680 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 18:39:45.409214 11211 raft_consensus.cc:1261] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Refusing update from remote peer 59329d7a2ed745dbb0cd77c85ae80522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:45.409515 11426 raft_consensus.cc:1261] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 FOLLOWER]: Refusing update from remote peer 59329d7a2ed745dbb0cd77c85ae80522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:45.410001 11543 consensus_queue.cc:1035] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:45.410215 11540 consensus_queue.cc:1035] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:45.420040 11552 mvcc.cc:205] Tried to move back new op lower bound from 7315626333828542464 to 7315626333636481024. Current Snapshot: MvccSnapshot[applied={T|T < 7315626333828542464}]
I20260806 18:39:45.431444 10780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:45.431638 10780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:45.434315 10780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810: Sending DeleteTablet for 3 replicas of tablet 95afcbfd37824c778fbf645397d1d0ef
I20260806 18:39:45.435232 10976 tablet_service.cc:1514] Processing DeleteTablet for tablet 95afcbfd37824c778fbf645397d1d0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:45 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:57600
I20260806 18:39:45.435444 11395 tablet_service.cc:1514] Processing DeleteTablet for tablet 95afcbfd37824c778fbf645397d1d0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:45 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:57426
I20260806 18:39:45.436095 11186 tablet_service.cc:1514] Processing DeleteTablet for tablet 95afcbfd37824c778fbf645397d1d0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:45 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:60508
I20260806 18:39:45.436756 11561 tablet_replica.cc:330] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: stopping tablet replica
I20260806 18:39:45.436738 10780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:39:45.437103 11561 raft_consensus.cc:2229] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 LEADER]: Raft consensus shutting down.
W20260806 18:39:45.437181 10780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:39:45.437462 11561 raft_consensus.cc:2258] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:45.439081 11561 ts_tablet_manager.cc:1905] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:45.442044 11563 tablet_replica.cc:330] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: stopping tablet replica
I20260806 18:39:45.442416 11395 tablet_service.cc:1467] Processing CreateTablet for tablet a8722cb7276545bc98a7ea5409dbdfd7 (DEFAULT_TABLE table=test-table [id=72ce9e5963004fc38f261c2977bef47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:45.442695 11395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8722cb7276545bc98a7ea5409dbdfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:45.443186 10976 tablet_service.cc:1467] Processing CreateTablet for tablet a8722cb7276545bc98a7ea5409dbdfd7 (DEFAULT_TABLE table=test-table [id=72ce9e5963004fc38f261c2977bef47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:45.443783 10976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8722cb7276545bc98a7ea5409dbdfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:45.444990 11530 tablet_bootstrap.cc:492] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: Bootstrap starting.
I20260806 18:39:45.446049 11530 tablet_bootstrap.cc:654] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:45.446223 11529 tablet_bootstrap.cc:492] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: Bootstrap starting.
I20260806 18:39:45.446421 11561 ts_tablet_manager.cc:1918] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:39:45.446519 11561 log.cc:1196] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/95afcbfd37824c778fbf645397d1d0ef
I20260806 18:39:45.446835 11561 ts_tablet_manager.cc:1939] T 95afcbfd37824c778fbf645397d1d0ef P 59329d7a2ed745dbb0cd77c85ae80522: Deleting consensus metadata
I20260806 18:39:45.447124 11529 tablet_bootstrap.cc:654] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:45.448107 10758 catalog_manager.cc:4841] TS 59329d7a2ed745dbb0cd77c85ae80522 (127.8.241.193:41493): tablet 95afcbfd37824c778fbf645397d1d0ef (table test-table [id=11000151ecf242b29c872148466da040]) successfully deleted
I20260806 18:39:45.442873 11186 tablet_service.cc:1467] Processing CreateTablet for tablet a8722cb7276545bc98a7ea5409dbdfd7 (DEFAULT_TABLE table=test-table [id=72ce9e5963004fc38f261c2977bef47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:45.448627 11186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8722cb7276545bc98a7ea5409dbdfd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:45.448928 11563 raft_consensus.cc:2229] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:45.449365 11563 raft_consensus.cc:2258] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:45.451229 11532 tablet_bootstrap.cc:492] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: Bootstrap starting.
I20260806 18:39:45.451344 11530 tablet_bootstrap.cc:492] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: No bootstrap required, opened a new log
I20260806 18:39:45.451428 11530 ts_tablet_manager.cc:1397] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260806 18:39:45.451985 11530 raft_consensus.cc:348] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.452107 11530 raft_consensus.cc:374] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:45.452142 11530 raft_consensus.cc:729] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb45b01783f46a3b8db4e29cb5a75c2, State: Initialized, Role: FOLLOWER
I20260806 18:39:45.452266 11532 tablet_bootstrap.cc:654] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:45.452275 11530 consensus_queue.cc:260] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.452343 11529 tablet_bootstrap.cc:492] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: No bootstrap required, opened a new log
I20260806 18:39:45.452453 11529 ts_tablet_manager.cc:1397] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260806 18:39:45.452545 11530 ts_tablet_manager.cc:1428] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 18:39:45.452549 11563 ts_tablet_manager.cc:1905] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:45.452934 11529 raft_consensus.cc:348] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.453076 11529 raft_consensus.cc:374] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:45.453111 11529 raft_consensus.cc:729] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59329d7a2ed745dbb0cd77c85ae80522, State: Initialized, Role: FOLLOWER
I20260806 18:39:45.453228 11529 consensus_queue.cc:260] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.453481 11529 ts_tablet_manager.cc:1428] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:39:45.454842 11562 tablet_replica.cc:330] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: stopping tablet replica
I20260806 18:39:45.455170 11563 ts_tablet_manager.cc:1918] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:39:45.455204 11562 raft_consensus.cc:2229] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:45.455276 11563 log.cc:1196] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/95afcbfd37824c778fbf645397d1d0ef
I20260806 18:39:45.455597 11563 ts_tablet_manager.cc:1939] T 95afcbfd37824c778fbf645397d1d0ef P e37c54693a2a4558bda143ff2ea5ef71: Deleting consensus metadata
I20260806 18:39:45.455585 11562 raft_consensus.cc:2258] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:45.456933 10758 catalog_manager.cc:4841] TS e37c54693a2a4558bda143ff2ea5ef71 (127.8.241.194:40115): tablet 95afcbfd37824c778fbf645397d1d0ef (table test-table [id=11000151ecf242b29c872148466da040]) successfully deleted
I20260806 18:39:45.457432 11562 ts_tablet_manager.cc:1905] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:45.457921 11532 tablet_bootstrap.cc:492] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: No bootstrap required, opened a new log
W20260806 18:39:45.457916 11497 tablet.cc:2369] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:39:45.458005 11532 ts_tablet_manager.cc:1397] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260806 18:39:45.458436 11532 raft_consensus.cc:348] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.458536 11532 raft_consensus.cc:374] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:45.458567 11532 raft_consensus.cc:729] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e37c54693a2a4558bda143ff2ea5ef71, State: Initialized, Role: FOLLOWER
I20260806 18:39:45.458688 11532 consensus_queue.cc:260] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.458940 11532 ts_tablet_manager.cc:1428] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:39:45.461376 11562 ts_tablet_manager.cc:1918] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:39:45.461490 11562 log.cc:1196] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/95afcbfd37824c778fbf645397d1d0ef
I20260806 18:39:45.461791 11562 ts_tablet_manager.cc:1939] T 95afcbfd37824c778fbf645397d1d0ef P 5cb45b01783f46a3b8db4e29cb5a75c2: Deleting consensus metadata
I20260806 18:39:45.463240 10757 catalog_manager.cc:4841] TS 5cb45b01783f46a3b8db4e29cb5a75c2 (127.8.241.195:34259): tablet 95afcbfd37824c778fbf645397d1d0ef (table test-table [id=11000151ecf242b29c872148466da040]) successfully deleted
W20260806 18:39:45.481931 11287 tablet.cc:2369] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:39:45.510452 11079 tablet.cc:2369] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:39:45.614463 11539 raft_consensus.cc:482] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:39:45.614639 11539 raft_consensus.cc:504] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.615732 11539 leader_election.cc:288] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59329d7a2ed745dbb0cd77c85ae80522 (127.8.241.193:41493), e37c54693a2a4558bda143ff2ea5ef71 (127.8.241.194:40115)
I20260806 18:39:45.619596 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.615975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:41493 (local address 127.8.241.195:54977)
0806 18:39:45.616106 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.616121 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.616226 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.616507 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.616511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.616532 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:45.616653 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.616662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.617575 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.617578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.618477 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.618487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.619189 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.619193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.619302 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.619330 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.619399 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.619477 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":8}
I20260806 18:39:45.619655 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.616069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:54977 (local address 127.8.241.193:41493)
0806 18:39:45.616191 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.616195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.616215 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:39:45.616331 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.616334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.616385 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.616481 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:45.616795 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.617463 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.618612 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.619084 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.619176 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.619222 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.619423 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.619522 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:39:45.619797 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.616415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:37337 (local address 127.8.241.194:40115)
0806 18:39:45.616666 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.616670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.616684 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:39:45.616735 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.616738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.616787 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.616850 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:45.617190 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.617748 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.618849 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.619280 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.619390 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.619434 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.619628 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.619697 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260806 18:39:45.619992 11567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.616251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:40115 (local address 127.8.241.195:37337)
0806 18:39:45.616588 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.616598 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.616669 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.616871 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.616874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.616890 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:45.617079 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.617087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.617866 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.617869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.618728 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.618739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.619404 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.619408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.619503 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.619527 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.619600 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.619790 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":89,"threads_started":1}
I20260806 18:39:45.620306 11002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8722cb7276545bc98a7ea5409dbdfd7" candidate_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59329d7a2ed745dbb0cd77c85ae80522" is_pre_election: true
I20260806 18:39:45.620432 11211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8722cb7276545bc98a7ea5409dbdfd7" candidate_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e37c54693a2a4558bda143ff2ea5ef71" is_pre_election: true
I20260806 18:39:45.620572 11211 raft_consensus.cc:2454] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cb45b01783f46a3b8db4e29cb5a75c2 in term 0.
I20260806 18:39:45.620585 11002 raft_consensus.cc:2454] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cb45b01783f46a3b8db4e29cb5a75c2 in term 0.
I20260806 18:39:45.620925 11324 leader_election.cc:302] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb45b01783f46a3b8db4e29cb5a75c2, e37c54693a2a4558bda143ff2ea5ef71; no voters:
I20260806 18:39:45.621210 11539 raft_consensus.cc:2781] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:39:45.621265 11539 raft_consensus.cc:482] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:39:45.621291 11539 raft_consensus.cc:3037] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:45.622273 11539 raft_consensus.cc:504] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.622605 11539 leader_election.cc:288] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [CANDIDATE]: Term 1 election: Requested vote from peers 59329d7a2ed745dbb0cd77c85ae80522 (127.8.241.193:41493), e37c54693a2a4558bda143ff2ea5ef71 (127.8.241.194:40115)
I20260806 18:39:45.622939 11002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8722cb7276545bc98a7ea5409dbdfd7" candidate_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59329d7a2ed745dbb0cd77c85ae80522"
I20260806 18:39:45.622985 11211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8722cb7276545bc98a7ea5409dbdfd7" candidate_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e37c54693a2a4558bda143ff2ea5ef71"
I20260806 18:39:45.623082 11211 raft_consensus.cc:3037] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:45.623080 11002 raft_consensus.cc:3037] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:45.624208 11002 raft_consensus.cc:2454] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cb45b01783f46a3b8db4e29cb5a75c2 in term 1.
I20260806 18:39:45.624208 11211 raft_consensus.cc:2454] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cb45b01783f46a3b8db4e29cb5a75c2 in term 1.
I20260806 18:39:45.624488 11324 leader_election.cc:302] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59329d7a2ed745dbb0cd77c85ae80522, 5cb45b01783f46a3b8db4e29cb5a75c2; no voters:
I20260806 18:39:45.624651 11539 raft_consensus.cc:2781] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:45.624840 11539 raft_consensus.cc:686] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 LEADER]: Becoming Leader. State: Replica: 5cb45b01783f46a3b8db4e29cb5a75c2, State: Running, Role: LEADER
I20260806 18:39:45.625149 11539 consensus_queue.cc:237] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.627389 10780 catalog_manager.cc:5495] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cb45b01783f46a3b8db4e29cb5a75c2 (127.8.241.195). New cstate: current_term: 1 leader_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } health_report { overall_health: HEALTHY } } }
Aug 06 18:39:45 dist-test-slave-k5rr krb5kdc[10720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041584, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:45.679140 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.670999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39242 (local address 127.8.241.254:45431)
0806 18:39:45.671120 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.671124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.671371 (+ 247us) server_negotiation.cc:407] Connection header received
0806 18:39:45.671504 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.671506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.671550 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.671625 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:45.672314 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.672817 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.673606 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.673806 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.673874 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.676221 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:45.676224 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:45.676226 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:45.676251 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:45.677602 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:45.678028 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:45.678030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:45.678032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:45.678099 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:45.678298 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:45.678300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:45.678301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:45.678482 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:45.678551 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.678866 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.678977 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":39}
I20260806 18:39:45.679173 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.670957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45431 (local address 127.0.0.1:39242)
0806 18:39:45.671283 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.671299 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.671420 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.671659 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.671661 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.672108 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:45.672212 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.672219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.672901 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.672904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.673520 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.673529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.673886 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.673889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.673998 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.674537 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:45.674550 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:45.676069 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:45.677696 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:45.677706 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:45.677712 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:45.677950 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:45.678163 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:45.678165 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:45.678166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:45.678238 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:45.678557 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:45.678562 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:45.678681 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.678771 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.678925 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":25,"thread_start_us":131,"threads_started":1}
I20260806 18:39:45.683476 10780 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:39242:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:39:45.684298 10780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:39:45.689482 10976 tablet_service.cc:1467] Processing CreateTablet for tablet 6efca57363384b349516ad5f6986d643 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=543aa9d09ed6427da74e4c3d171ded84]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:39:45.689484 11186 tablet_service.cc:1467] Processing CreateTablet for tablet 6efca57363384b349516ad5f6986d643 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=543aa9d09ed6427da74e4c3d171ded84]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:39:45.689568 11395 tablet_service.cc:1467] Processing CreateTablet for tablet 6efca57363384b349516ad5f6986d643 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=543aa9d09ed6427da74e4c3d171ded84]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:39:45.689930 10976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6efca57363384b349516ad5f6986d643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:45.689929 11395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6efca57363384b349516ad5f6986d643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:45.689929 11186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6efca57363384b349516ad5f6986d643. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:45.692535 11529 tablet_bootstrap.cc:492] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: Bootstrap starting.
I20260806 18:39:45.692960 11530 tablet_bootstrap.cc:492] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: Bootstrap starting.
I20260806 18:39:45.693296 11532 tablet_bootstrap.cc:492] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: Bootstrap starting.
I20260806 18:39:45.693573 11529 tablet_bootstrap.cc:654] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:45.693933 11530 tablet_bootstrap.cc:654] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:45.694217 11532 tablet_bootstrap.cc:654] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:45.695222 11530 tablet_bootstrap.cc:492] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: No bootstrap required, opened a new log
I20260806 18:39:45.695310 11530 ts_tablet_manager.cc:1397] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:39:45.695555 11529 tablet_bootstrap.cc:492] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: No bootstrap required, opened a new log
I20260806 18:39:45.695554 11532 tablet_bootstrap.cc:492] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: No bootstrap required, opened a new log
I20260806 18:39:45.695626 11532 ts_tablet_manager.cc:1397] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:39:45.695623 11529 ts_tablet_manager.cc:1397] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:39:45.696058 11532 raft_consensus.cc:348] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.696058 11530 raft_consensus.cc:348] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.696063 11529 raft_consensus.cc:348] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.696175 11530 raft_consensus.cc:374] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:45.696175 11532 raft_consensus.cc:374] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:45.696185 11529 raft_consensus.cc:374] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:45.696208 11530 raft_consensus.cc:729] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb45b01783f46a3b8db4e29cb5a75c2, State: Initialized, Role: FOLLOWER
I20260806 18:39:45.696208 11532 raft_consensus.cc:729] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e37c54693a2a4558bda143ff2ea5ef71, State: Initialized, Role: FOLLOWER
I20260806 18:39:45.696219 11529 raft_consensus.cc:729] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59329d7a2ed745dbb0cd77c85ae80522, State: Initialized, Role: FOLLOWER
I20260806 18:39:45.696334 11530 consensus_queue.cc:260] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.696352 11529 consensus_queue.cc:260] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.696334 11532 consensus_queue.cc:260] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.696632 11532 ts_tablet_manager.cc:1428] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:39:45.696633 11529 ts_tablet_manager.cc:1428] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:39:45.696751 11539 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:45.696887 11539 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:45.696823 11556 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:45.696799 11538 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:45.696949 11556 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:45.696957 11538 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:45.697197 11530 ts_tablet_manager.cc:1428] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:39:45.932678 11538 raft_consensus.cc:482] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:39:45.932839 11538 raft_consensus.cc:504] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.934088 11538 leader_election.cc:288] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59329d7a2ed745dbb0cd77c85ae80522 (127.8.241.193:41493), 5cb45b01783f46a3b8db4e29cb5a75c2 (127.8.241.195:34259)
I20260806 18:39:45.938153 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.934431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:34343 (local address 127.8.241.193:41493)
0806 18:39:45.934578 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.934583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.934777 (+ 194us) server_negotiation.cc:407] Connection header received
0806 18:39:45.934914 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.934916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.934966 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.935029 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:45.935298 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.935865 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.937160 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.937560 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.937675 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.937735 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.937943 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.938026 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:39:45.938167 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.934479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:32931 (local address 127.8.241.195:34259)
0806 18:39:45.934696 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.934700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.934864 (+ 164us) server_negotiation.cc:407] Connection header received
0806 18:39:45.934990 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.934993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.935040 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.935112 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:45.935366 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.935956 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.937154 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.937574 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.937684 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.937744 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.937941 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.938015 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":60,"threads_started":1}
I20260806 18:39:45.938182 11592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.934364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:34259 (local address 127.8.241.194:32931)
0806 18:39:45.934783 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.934794 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.934895 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.935128 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.935130 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.935146 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:45.935263 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.935269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.936082 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.936085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.937036 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.937050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.937685 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.937688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.937816 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.937849 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.937914 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.937998 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":29,"thread_start_us":84,"threads_started":1}
I20260806 18:39:45.938181 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.934323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:41493 (local address 127.8.241.194:34343)
0806 18:39:45.934690 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.934703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.934812 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.935053 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.935056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.935072 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:45.935196 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.935201 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.935972 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.935975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.936987 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.936998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.937681 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.937685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.937818 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.937851 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.937916 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.937993 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":52,"thread_start_us":110,"threads_started":1}
I20260806 18:39:45.938668 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efca57363384b349516ad5f6986d643" candidate_uuid: "e37c54693a2a4558bda143ff2ea5ef71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" is_pre_election: true
I20260806 18:39:45.938827 11426 raft_consensus.cc:2454] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e37c54693a2a4558bda143ff2ea5ef71 in term 0.
I20260806 18:39:45.939003 11002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efca57363384b349516ad5f6986d643" candidate_uuid: "e37c54693a2a4558bda143ff2ea5ef71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59329d7a2ed745dbb0cd77c85ae80522" is_pre_election: true
I20260806 18:39:45.939165 11002 raft_consensus.cc:2454] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e37c54693a2a4558bda143ff2ea5ef71 in term 0.
I20260806 18:39:45.939218 11109 leader_election.cc:302] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb45b01783f46a3b8db4e29cb5a75c2, e37c54693a2a4558bda143ff2ea5ef71; no voters:
I20260806 18:39:45.939503 11538 raft_consensus.cc:2781] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:39:45.939568 11538 raft_consensus.cc:482] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:39:45.939604 11538 raft_consensus.cc:3037] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:45.940618 11538 raft_consensus.cc:504] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.940953 11538 leader_election.cc:288] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [CANDIDATE]: Term 1 election: Requested vote from peers 59329d7a2ed745dbb0cd77c85ae80522 (127.8.241.193:41493), 5cb45b01783f46a3b8db4e29cb5a75c2 (127.8.241.195:34259)
I20260806 18:39:45.941316 11002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efca57363384b349516ad5f6986d643" candidate_uuid: "e37c54693a2a4558bda143ff2ea5ef71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59329d7a2ed745dbb0cd77c85ae80522"
I20260806 18:39:45.941355 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6efca57363384b349516ad5f6986d643" candidate_uuid: "e37c54693a2a4558bda143ff2ea5ef71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2"
I20260806 18:39:45.941462 11426 raft_consensus.cc:3037] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:45.941469 11002 raft_consensus.cc:3037] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:45.942616 11426 raft_consensus.cc:2454] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e37c54693a2a4558bda143ff2ea5ef71 in term 1.
I20260806 18:39:45.942616 11002 raft_consensus.cc:2454] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e37c54693a2a4558bda143ff2ea5ef71 in term 1.
I20260806 18:39:45.942921 11109 leader_election.cc:302] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb45b01783f46a3b8db4e29cb5a75c2, e37c54693a2a4558bda143ff2ea5ef71; no voters:
I20260806 18:39:45.943082 11538 raft_consensus.cc:2781] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:45.943279 11538 raft_consensus.cc:686] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [term 1 LEADER]: Becoming Leader. State: Replica: e37c54693a2a4558bda143ff2ea5ef71, State: Running, Role: LEADER
I20260806 18:39:45.943536 11538 consensus_queue.cc:237] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } }
I20260806 18:39:45.944938 11598 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: TxnStatusTablet state changed. Reason: New leader e37c54693a2a4558bda143ff2ea5ef71. Latest consensus state: current_term: 1 leader_uuid: "e37c54693a2a4558bda143ff2ea5ef71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:45.945091 11598 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:39:45.945454 11600 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:39:45.945731 11600 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:39:45.946337 10780 catalog_manager.cc:5495] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 reported cstate change: term changed from 0 to 1, leader changed from <none> to e37c54693a2a4558bda143ff2ea5ef71 (127.8.241.194). New cstate: current_term: 1 leader_uuid: "e37c54693a2a4558bda143ff2ea5ef71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } health_report { overall_health: UNKNOWN } } }
I20260806 18:39:45.997761 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.993050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60516 (local address 127.8.241.194:40115)
0806 18:39:45.993175 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:45.993179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:45.993326 (+ 147us) server_negotiation.cc:407] Connection header received
0806 18:39:45.993521 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:45.993525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:45.993587 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:45.993692 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:45.993944 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.994697 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.995919 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.996272 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.996366 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.996695 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:45.997357 (+ 662us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:45.997468 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:45.997559 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 18:39:45.997620 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:39:45.997809 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:45.992933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:40115 (local address 127.0.0.1:60516)
0806 18:39:45.993247 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:45.993262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:45.993384 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:45.993714 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:45.993718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:45.993726 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:39:45.993845 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.993851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.994812 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.994815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.995803 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:45.995815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:45.996382 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:45.996385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:45.996478 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:45.996548 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:39:45.997463 (+ 915us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:39:45.997481 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:39:45.997548 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:39:45.997658 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260806 18:39:46.001653 11426 raft_consensus.cc:1261] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 FOLLOWER]: Refusing update from remote peer e37c54693a2a4558bda143ff2ea5ef71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:46.001782 11002 raft_consensus.cc:1261] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 FOLLOWER]: Refusing update from remote peer e37c54693a2a4558bda143ff2ea5ef71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:46.002259 11598 consensus_queue.cc:1035] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:46.002529 11538 consensus_queue.cc:1035] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:46.004000 11539 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: TxnStatusTablet state changed. Reason: New leader e37c54693a2a4558bda143ff2ea5ef71. Latest consensus state: current_term: 1 leader_uuid: "e37c54693a2a4558bda143ff2ea5ef71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:46.004127 11539 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:46.004426 11556 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: TxnStatusTablet state changed. Reason: New leader e37c54693a2a4558bda143ff2ea5ef71. Latest consensus state: current_term: 1 leader_uuid: "e37c54693a2a4558bda143ff2ea5ef71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:46.004549 11556 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:46.007040 11538 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e37c54693a2a4558bda143ff2ea5ef71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:46.007165 11538 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:39:46.007419 11598 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e37c54693a2a4558bda143ff2ea5ef71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:46.007521 11598 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P e37c54693a2a4558bda143ff2ea5ef71: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:39:46.008301 11556 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e37c54693a2a4558bda143ff2ea5ef71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:46.008417 11556 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P 59329d7a2ed745dbb0cd77c85ae80522: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:46.009438 11609 mvcc.cc:205] Tried to move back new op lower bound from 7315626336256557056 to 7315626336027181056. Current Snapshot: MvccSnapshot[applied={T|T < 7315626336256557056}]
I20260806 18:39:46.009384 11539 tablet_replica.cc:439] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e37c54693a2a4558bda143ff2ea5ef71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 } } peers { permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 } } peers { permanent_uuid: "5cb45b01783f46a3b8db4e29cb5a75c2" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34259 } } }
I20260806 18:39:46.009563 11539 tablet_replica.cc:442] T 6efca57363384b349516ad5f6986d643 P 5cb45b01783f46a3b8db4e29cb5a75c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:46.010299 11612 mvcc.cc:205] Tried to move back new op lower bound from 7315626336256557056 to 7315626336027181056. Current Snapshot: MvccSnapshot[applied={T|T < 7315626336256557056 or (T in {7315626336256557056})}]
I20260806 18:39:46.019615 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:46.015736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.8.241.195:34259)
0806 18:39:46.015864 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:46.015868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:46.015969 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:39:46.016129 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:46.016132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:46.016188 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:46.016275 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:46.016512 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.017040 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.018199 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.018408 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.018484 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:46.018759 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:46.019239 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:46.019322 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:46.019409 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 18:39:46.019474 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:39:46.030886 11621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:46.027325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:40115 (local address 127.8.241.195:50305)
0806 18:39:46.027699 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:46.027712 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:46.027803 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:46.028037 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:46.028039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:46.028045 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:39:46.028131 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:46.028135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.029112 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.029114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:46.029720 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:46.029729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.030280 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.030283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:46.030341 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:46.030389 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:39:46.030653 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:39:46.030657 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:39:46.030699 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 18:39:46.030759 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":110,"threads_started":1}
I20260806 18:39:46.030898 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:46.027416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:50305 (local address 127.8.241.194:40115)
0806 18:39:46.027533 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:46.027536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:46.027789 (+ 253us) server_negotiation.cc:407] Connection header received
0806 18:39:46.027882 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:46.027885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:46.027949 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:46.028021 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:46.028211 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.028994 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.029821 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.030166 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.030264 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:46.030503 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:46.030586 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:46.030652 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:46.030710 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 18:39:46.030762 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:39:46.038010 11211 raft_consensus.cc:1261] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Refusing update from remote peer 5cb45b01783f46a3b8db4e29cb5a75c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:46.038004 11002 raft_consensus.cc:1261] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 FOLLOWER]: Refusing update from remote peer 5cb45b01783f46a3b8db4e29cb5a75c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:46.038563 11539 consensus_queue.cc:1035] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59329d7a2ed745dbb0cd77c85ae80522" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:46.038763 11571 consensus_queue.cc:1035] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e37c54693a2a4558bda143ff2ea5ef71" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:46.062775 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:46.058857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:41935 (local address 127.8.241.195:34259)
0806 18:39:46.058977 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:46.058982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:46.059180 (+ 198us) server_negotiation.cc:407] Connection header received
0806 18:39:46.059304 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:46.059306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:46.059350 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:46.059417 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:46.059715 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.060488 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.061594 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.061906 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.062027 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:46.062298 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:46.062388 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:46.062457 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:46.062561 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 18:39:46.062617 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:39:46.062811 11628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:46.058735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:34259 (local address 127.8.241.194:41935)
0806 18:39:46.059097 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:46.059113 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:46.059225 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:46.059452 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:46.059455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:46.059465 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:39:46.059609 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:46.059616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.060694 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.060697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:46.061474 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:46.061486 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.062036 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.062038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:46.062110 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:46.062159 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:39:46.062478 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:39:46.062485 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:39:46.062539 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 18:39:46.062628 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":121,"threads_started":1}
I20260806 18:39:46.087812 10778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:46.088028 10778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:46.090559 10778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d4b3cacea4e424aadebd520ea33f810: Sending DeleteTablet for 3 replicas of tablet a8722cb7276545bc98a7ea5409dbdfd7
I20260806 18:39:46.091284 11395 tablet_service.cc:1514] Processing DeleteTablet for tablet a8722cb7276545bc98a7ea5409dbdfd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:46 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:57426
I20260806 18:39:46.091413 10976 tablet_service.cc:1514] Processing DeleteTablet for tablet a8722cb7276545bc98a7ea5409dbdfd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:46 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:57600
I20260806 18:39:46.091795 11186 tablet_service.cc:1514] Processing DeleteTablet for tablet a8722cb7276545bc98a7ea5409dbdfd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:46 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:60508
I20260806 18:39:46.092303 11634 tablet_replica.cc:330] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: stopping tablet replica
I20260806 18:39:46.092542 11635 tablet_replica.cc:330] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: stopping tablet replica
I20260806 18:39:46.092739 11634 raft_consensus.cc:2229] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:46.092923 11635 raft_consensus.cc:2229] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:46.093127 11635 raft_consensus.cc:2258] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:46.093307 11634 raft_consensus.cc:2258] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:46.094365 11635 ts_tablet_manager.cc:1905] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:46.094904 11634 ts_tablet_manager.cc:1905] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:46.098097 11634 ts_tablet_manager.cc:1918] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
Aug 06 18:39:46 dist-test-slave-k5rr krb5kdc[10720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041585, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:39:46.100768 11632 tablet_replica.cc:330] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: stopping tablet replica
I20260806 18:39:46.100975 11632 raft_consensus.cc:2229] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:39:46.101389 11632 raft_consensus.cc:2258] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:46.102157 11632 ts_tablet_manager.cc:1905] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:46.106115 11635 ts_tablet_manager.cc:1918] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:39:46.106230 11635 log.cc:1196] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/a8722cb7276545bc98a7ea5409dbdfd7
I20260806 18:39:46.106499 11635 ts_tablet_manager.cc:1939] T a8722cb7276545bc98a7ea5409dbdfd7 P e37c54693a2a4558bda143ff2ea5ef71: Deleting consensus metadata
I20260806 18:39:46.105432 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:46.092933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57612 (local address 127.8.241.193:41493)
0806 18:39:46.093084 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:46.093089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:46.093582 (+ 493us) server_negotiation.cc:407] Connection header received
0806 18:39:46.093751 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:46.093754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:46.093797 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:46.093845 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:46.094768 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.095551 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.096740 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.097099 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.097172 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:46.101182 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:46.101195 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:46.101214 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:46.101242 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:46.103234 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:46.103742 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:46.103750 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:46.103752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:46.103849 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:46.104177 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:46.104181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:46.104183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:46.104508 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:46.104683 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:46.105110 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 18:39:46.105261 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:39:46.107432 10758 catalog_manager.cc:4841] TS e37c54693a2a4558bda143ff2ea5ef71 (127.8.241.194:40115): tablet a8722cb7276545bc98a7ea5409dbdfd7 (table test-table [id=72ce9e5963004fc38f261c2977bef47a]) successfully deleted
W20260806 18:39:46.107615 11066 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57612
I20260806 18:39:46.107944 11634 log.cc:1196] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/a8722cb7276545bc98a7ea5409dbdfd7
I20260806 18:39:46.108222 11634 ts_tablet_manager.cc:1939] T a8722cb7276545bc98a7ea5409dbdfd7 P 59329d7a2ed745dbb0cd77c85ae80522: Deleting consensus metadata
I20260806 18:39:46.108954 10758 catalog_manager.cc:4841] TS 59329d7a2ed745dbb0cd77c85ae80522 (127.8.241.193:41493): tablet a8722cb7276545bc98a7ea5409dbdfd7 (table test-table [id=72ce9e5963004fc38f261c2977bef47a]) successfully deleted
I20260806 18:39:46.104995 11632 ts_tablet_manager.cc:1918] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:39:46.109309 11632 log.cc:1196] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/a8722cb7276545bc98a7ea5409dbdfd7
I20260806 18:39:46.109579 11632 ts_tablet_manager.cc:1939] T a8722cb7276545bc98a7ea5409dbdfd7 P 5cb45b01783f46a3b8db4e29cb5a75c2: Deleting consensus metadata
I20260806 18:39:46.110287 10757 catalog_manager.cc:4841] TS 5cb45b01783f46a3b8db4e29cb5a75c2 (127.8.241.195:34259): tablet a8722cb7276545bc98a7ea5409dbdfd7 (table test-table [id=72ce9e5963004fc38f261c2977bef47a]) successfully deleted
I20260806 18:39:46.120756 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:46.110677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.8.241.254:45431)
0806 18:39:46.110805 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:46.110809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:46.111026 (+ 217us) server_negotiation.cc:407] Connection header received
0806 18:39:46.111137 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:46.111140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:46.111191 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:46.111275 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:46.112052 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.112907 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.114207 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:46.114618 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:46.114724 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:46.116572 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:46.116576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:46.116579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:46.116611 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:46.118295 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:46.118894 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:46.118898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:46.118900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:46.119078 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:46.119467 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:46.119471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:46.119473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:46.119718 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:46.119886 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:46.120482 (+ 596us) server_negotiation.cc:299] Negotiation successful
0806 18:39:46.120598 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 18:39:46.121255 10778 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39248
I20260806 18:39:46.122689 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10882
I20260806 18:39:46.139247 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 11087
I20260806 18:39:46.158515 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 11298
I20260806 18:39:46.171581 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10736
2026-08-06T18:39:46Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3245 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:46 dist-test-slave-k5rr krb5kdc[11643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:46 dist-test-slave-k5rr krb5kdc[11643](info): set up 2 sockets
Aug 06 18:39:46 dist-test-slave-k5rr krb5kdc[11643](info): commencing operation
krb5kdc: starting...
W20260806 18:39:47.316759 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:39:47 dist-test-slave-k5rr krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:39:47.444316 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36237
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:36237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:47.564446 11664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:47.564735 11664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:47.564793 11664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:47.568799 11664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:47.568866 11664 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:39:47.568890 11664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:47.568920 11664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:47.568943 11664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:47.568971 11664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:47.574373 11664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:36237
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:47.575508 11664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:47.576581 11664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:47.585284 11677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:47.585291 11672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:47.585314 11674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:47.585510 11664 server_base.cc:1034] running on GCE node
I20260806 18:39:47.586141 11664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:47.587344 11664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:47.588517 11664 hybrid_clock.cc:648] HybridClock initialized: now 1786041587588494 us; error 35 us; skew 500 ppm
Aug 06 18:39:47 dist-test-slave-k5rr krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041587, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:47.591216 11664 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:39:47.592166 11664 webserver.cc:460] Webserver started at http://127.8.241.254:33141/ using document root <none> and password file <none>
I20260806 18:39:47.592712 11664 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:47.592769 11664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:47.592949 11664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:47.595091 11664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "7768074f117e497d821a767accea3aff"
format_stamp: "Formatted at 2026-08-06 18:39:47 on dist-test-slave-k5rr"
server_key: "3902cf3cee23d55a7d3100d77bc25916"
server_key_iv: "c8d10752790541ecd95d166ba1126ea2"
server_key_version: "encryptionkey@0"
I20260806 18:39:47.595624 11664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "7768074f117e497d821a767accea3aff"
format_stamp: "Formatted at 2026-08-06 18:39:47 on dist-test-slave-k5rr"
server_key: "3902cf3cee23d55a7d3100d77bc25916"
server_key_iv: "c8d10752790541ecd95d166ba1126ea2"
server_key_version: "encryptionkey@0"
I20260806 18:39:47.599416 11664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 18:39:47.601941 11688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:47.603185 11664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:39:47.603309 11664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "7768074f117e497d821a767accea3aff"
format_stamp: "Formatted at 2026-08-06 18:39:47 on dist-test-slave-k5rr"
server_key: "3902cf3cee23d55a7d3100d77bc25916"
server_key_iv: "c8d10752790541ecd95d166ba1126ea2"
server_key_version: "encryptionkey@0"
I20260806 18:39:47.603417 11664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:47.615520 11664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:47.618556 11664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:47.618722 11664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:47.626585 11664 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:36237
I20260806 18:39:47.626597 11788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:36237 every 8 connection(s)
I20260806 18:39:47.627586 11664 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:47.630636 11789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:47.631402 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 11664
I20260806 18:39:47.631508 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:39:47.631769 9159 external_mini_cluster.cc:1428] Setting key 1328e516c409ff70571b2afd51e8733c
I20260806 18:39:47.637095 11789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff: Bootstrap starting.
I20260806 18:39:47.639657 11789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:47.640565 11789 log.cc:824] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:39:47 dist-test-slave-k5rr krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:47.642854 11789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff: No bootstrap required, opened a new log
I20260806 18:39:47.646661 11789 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7768074f117e497d821a767accea3aff" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36237 } }
I20260806 18:39:47.646905 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:47.633284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40548 (local address 127.8.241.254:36237)
0806 18:39:47.633788 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:47.633803 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:47.633841 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:39:47.634508 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:47.634527 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:47.634869 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:47.635199 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:47.636023 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.637079 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.637945 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.638445 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.638600 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:47.641353 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:47.641360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:47.641377 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:47.641416 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:47.643782 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:47.644329 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:47.644334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:47.644339 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:47.644459 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:47.644728 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:47.644730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:47.644732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:47.645250 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:47.645426 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:47.645719 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 18:39:47.646068 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":148,"threads_started":1}
I20260806 18:39:47.646925 11789 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:47.646998 11789 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7768074f117e497d821a767accea3aff, State: Initialized, Role: FOLLOWER
I20260806 18:39:47.647461 11789 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7768074f117e497d821a767accea3aff" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36237 } }
I20260806 18:39:47.647601 11789 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:47.647689 11789 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:47.647794 11789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:47.648763 11789 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7768074f117e497d821a767accea3aff" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36237 } }
I20260806 18:39:47.649152 11789 leader_election.cc:302] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7768074f117e497d821a767accea3aff; no voters:
I20260806 18:39:47.649498 11789 leader_election.cc:288] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:47.649585 11799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:47.649821 11799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [term 1 LEADER]: Becoming Leader. State: Replica: 7768074f117e497d821a767accea3aff, State: Running, Role: LEADER
I20260806 18:39:47.650168 11799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7768074f117e497d821a767accea3aff" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36237 } }
I20260806 18:39:47.650761 11789 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:47.652002 11801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7768074f117e497d821a767accea3aff. Latest consensus state: current_term: 1 leader_uuid: "7768074f117e497d821a767accea3aff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7768074f117e497d821a767accea3aff" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36237 } } }
I20260806 18:39:47.652031 11800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7768074f117e497d821a767accea3aff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7768074f117e497d821a767accea3aff" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36237 } } }
I20260806 18:39:47.652184 11801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:47.652245 11800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:47.653429 11808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:47.656267 11808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:47.661899 11808 catalog_manager.cc:1347] Generated new cluster ID: df627600e31f47459a87b6fd6f3b3c27
I20260806 18:39:47.661989 11808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:47.686422 11808 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:47.687477 11808 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:47.693292 11808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff: Generated new TSK 0
I20260806 18:39:47.694059 11808 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:39:47.733318 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:36237
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:47.881191 11823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:47.881510 11823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:47.881563 11823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:47.885828 11823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:47.885914 11823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:47.885960 11823 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:39:47.885989 11823 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:39:47.886041 11823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:39:47.891669 11823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.241.254:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:47.892854 11823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:47.893956 11823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:47.903630 11833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:47.903677 11829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:47.903658 11830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:47.903970 11823 server_base.cc:1034] running on GCE node
I20260806 18:39:47.904428 11823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:47.905213 11823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:47.906383 11823 hybrid_clock.cc:648] HybridClock initialized: now 1786041587906352 us; error 43 us; skew 500 ppm
Aug 06 18:39:47 dist-test-slave-k5rr krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041587, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:47.915767 11823 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:39:47.916924 11823 webserver.cc:460] Webserver started at http://127.8.241.193:34273/ using document root <none> and password file <none>
I20260806 18:39:47.917634 11823 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:47.917717 11823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:47.917954 11823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:47.920197 11823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "1f1ae74e03424b7786debc2b73ba5b45"
format_stamp: "Formatted at 2026-08-06 18:39:47 on dist-test-slave-k5rr"
server_key: "74cf9e687f4f523cc9cae7b4487d7817"
server_key_iv: "03cab131ac6489c06bf302025a62a19b"
server_key_version: "encryptionkey@0"
I20260806 18:39:47.920825 11823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "1f1ae74e03424b7786debc2b73ba5b45"
format_stamp: "Formatted at 2026-08-06 18:39:47 on dist-test-slave-k5rr"
server_key: "74cf9e687f4f523cc9cae7b4487d7817"
server_key_iv: "03cab131ac6489c06bf302025a62a19b"
server_key_version: "encryptionkey@0"
I20260806 18:39:47.924866 11823 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 18:39:47.927568 11844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:47.928846 11823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:39:47.928965 11823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "1f1ae74e03424b7786debc2b73ba5b45"
format_stamp: "Formatted at 2026-08-06 18:39:47 on dist-test-slave-k5rr"
server_key: "74cf9e687f4f523cc9cae7b4487d7817"
server_key_iv: "03cab131ac6489c06bf302025a62a19b"
server_key_version: "encryptionkey@0"
I20260806 18:39:47.929103 11823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:47.946655 11823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:47.949846 11823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:47.950054 11823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:47.951802 11823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:47.951859 11823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:47.951917 11823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:47.951944 11823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:39:47 dist-test-slave-k5rr krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041587, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:47.962891 11823 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:44857
I20260806 18:39:47.964068 11823 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:39:47.964303 12028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:44857 every 8 connection(s)
I20260806 18:39:47.966254 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:47.952722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:41511 (local address 127.8.241.254:36237)
0806 18:39:47.952849 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:47.952854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:47.954115 (+ 1261us) server_negotiation.cc:407] Connection header received
0806 18:39:47.955223 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:47.955226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:47.955293 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:47.955392 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:47.956807 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.957404 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.959149 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.959398 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.959498 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:47.962399 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:47.962402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:47.962404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:47.962430 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:47.963958 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:47.964601 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:47.964605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:47.964607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:47.964702 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:47.965141 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:47.965145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:47.965147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:47.965345 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:47.965437 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:47.965947 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 18:39:47.966088 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:39:47.967020 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:47.953426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36237 (local address 127.8.241.193:41511)
0806 18:39:47.953944 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:47.954007 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:47.954977 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:47.955631 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:47.955641 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:47.956073 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:47.956626 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:47.956642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.957545 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.957549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:47.959005 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:47.959020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.959582 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.959587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:47.959809 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:47.960416 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:47.960440 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:47.962226 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:47.964076 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:47.964081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:47.964094 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:47.964474 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:47.964811 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:47.964814 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:47.964816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:47.965045 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:47.965451 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:47.965458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:47.965702 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 18:39:47.965865 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 18:39:47.966216 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":146,"threads_started":1}
I20260806 18:39:47.968333 12030 heartbeater.cc:347] Connected to a master server at 127.8.241.254:36237
I20260806 18:39:47.968492 12030 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:47.968984 12030 heartbeater.cc:511] Master 127.8.241.254:36237 requested a full tablet report, sending...
I20260806 18:39:47.970597 11708 ts_manager.cc:194] Registered new tserver with Master: 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857)
I20260806 18:39:47.971698 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 11823
I20260806 18:39:47.971807 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:39:47.972081 9159 external_mini_cluster.cc:1428] Setting key 5ee5b44255657816e3e0cd9e6257523d
I20260806 18:39:47.972168 11708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:41511
I20260806 18:39:47.977865 12036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:47.970379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36237 (local address 127.8.241.193:35417)
0806 18:39:47.970683 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:47.970694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:47.970823 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:47.971060 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:47.971062 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:47.971236 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:47.971336 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:47.971341 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.972401 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.972405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:47.973231 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:47.973242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.973789 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.973793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:47.973883 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:47.974427 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:47.974441 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:47.974839 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:47.976422 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:47.976426 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:47.976428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:47.976627 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:47.976880 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:47.976883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:47.976885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:47.976973 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:47.977420 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:47.977423 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:47.977524 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:39:47.977604 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:39:47.977721 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":92,"threads_started":1}
I20260806 18:39:47.977952 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:47.970473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:35417 (local address 127.8.241.254:36237)
0806 18:39:47.970595 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:47.970600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:47.970754 (+ 154us) server_negotiation.cc:407] Connection header received
0806 18:39:47.970916 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:47.970919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:47.970964 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:47.971046 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:47.971449 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.972242 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.973340 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:47.973572 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:47.973665 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:47.974979 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:47.974982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:47.974984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:47.975006 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:47.976314 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:47.976716 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:47.976719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:47.976721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:47.976797 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:47.977106 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:47.977109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:47.977111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:47.977325 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:47.977419 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:47.977678 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 18:39:47.977795 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:39:48.001379 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:36237
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:48.120951 12040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:48.121282 12040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:48.121361 12040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:48.125427 12040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:48.125507 12040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:48.125573 12040 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:39:48.125605 12040 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:39:48.125660 12040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:39:48.131202 12040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.241.254:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:48.132424 12040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:48.133625 12040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:48.143824 12047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:48.143824 12055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:48.143842 12049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:48.144098 12040 server_base.cc:1034] running on GCE node
I20260806 18:39:48.144492 12040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:48.145206 12040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:48.146401 12040 hybrid_clock.cc:648] HybridClock initialized: now 1786041588146367 us; error 42 us; skew 500 ppm
Aug 06 18:39:48 dist-test-slave-k5rr krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041588, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:48.149346 12040 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:39:48.150348 12040 webserver.cc:460] Webserver started at http://127.8.241.194:40259/ using document root <none> and password file <none>
I20260806 18:39:48.150939 12040 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:48.151005 12040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:48.151185 12040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:48.153503 12040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "7a844e184a27450c9b02ffb3573c705e"
format_stamp: "Formatted at 2026-08-06 18:39:48 on dist-test-slave-k5rr"
server_key: "5065c3ea92b1dcd982d1b5480d24eed3"
server_key_iv: "336f021980e70cd72ad402af2ed5294f"
server_key_version: "encryptionkey@0"
I20260806 18:39:48.154058 12040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "7a844e184a27450c9b02ffb3573c705e"
format_stamp: "Formatted at 2026-08-06 18:39:48 on dist-test-slave-k5rr"
server_key: "5065c3ea92b1dcd982d1b5480d24eed3"
server_key_iv: "336f021980e70cd72ad402af2ed5294f"
server_key_version: "encryptionkey@0"
I20260806 18:39:48.157918 12040 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:39:48.160360 12065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:48.161701 12040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:39:48.161830 12040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "7a844e184a27450c9b02ffb3573c705e"
format_stamp: "Formatted at 2026-08-06 18:39:48 on dist-test-slave-k5rr"
server_key: "5065c3ea92b1dcd982d1b5480d24eed3"
server_key_iv: "336f021980e70cd72ad402af2ed5294f"
server_key_version: "encryptionkey@0"
I20260806 18:39:48.161942 12040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:48.178587 12040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:48.181635 12040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:48.181828 12040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:48.183429 12040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:48.183497 12040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:48.183555 12040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:48.183584 12040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:39:48 dist-test-slave-k5rr krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041588, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:48.193993 12040 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:35197
I20260806 18:39:48.194036 12244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:35197 every 8 connection(s)
I20260806 18:39:48.195039 12040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:39:48.197964 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.184314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:52921 (local address 127.8.241.254:36237)
0806 18:39:48.184427 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.184431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.185618 (+ 1187us) server_negotiation.cc:407] Connection header received
0806 18:39:48.186765 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.186769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.186836 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.186939 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:48.188531 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.189380 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.190594 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.190918 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.191047 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.194210 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:48.194214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:48.194218 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:48.194247 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:48.195825 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.196349 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.196353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.196354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.196441 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.196780 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.196784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.196785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.196998 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:48.197122 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.197637 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.197783 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:39:48.198874 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.184897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36237 (local address 127.8.241.194:52921)
0806 18:39:48.185472 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.185518 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.186460 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.187176 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.187186 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.187593 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:48.188340 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.188363 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.189522 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.189526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.190479 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.190490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.191120 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.191124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.191282 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.191906 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:48.191931 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:48.194017 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:48.195944 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.195950 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.195968 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.196248 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.196532 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.196535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.196537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.196687 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.197122 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:48.197130 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:48.197386 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.197573 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.197924 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":146,"threads_started":1}
I20260806 18:39:48.198966 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 12040
I20260806 18:39:48.199108 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:39:48.199388 9159 external_mini_cluster.cc:1428] Setting key 7a4fe9c0b89bf6f3a8fb9f62270ec4f9
I20260806 18:39:48.200285 12246 heartbeater.cc:347] Connected to a master server at 127.8.241.254:36237
I20260806 18:39:48.200455 12246 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:48.201139 12246 heartbeater.cc:511] Master 127.8.241.254:36237 requested a full tablet report, sending...
I20260806 18:39:48.202337 11708 ts_manager.cc:194] Registered new tserver with Master: 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197)
I20260806 18:39:48.202916 11708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:52921
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:39:48.212226 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.202450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36237 (local address 127.8.241.194:38883)
0806 18:39:48.203066 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.203080 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.203182 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.205385 (+ 2203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.205389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.205626 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:48.205755 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.205763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.206548 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.206552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.207464 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.207478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.207932 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.207936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.208040 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.208539 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:48.208568 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:48.209080 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:48.210647 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.210650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.210653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.210940 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.211188 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.211191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.211193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.211287 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.211739 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:48.211743 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:48.211851 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.211944 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.212072 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":86,"threads_started":1}
I20260806 18:39:48.212309 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.202771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:38883 (local address 127.8.241.254:36237)
0806 18:39:48.202883 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.202887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.203141 (+ 254us) server_negotiation.cc:407] Connection header received
0806 18:39:48.205085 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.205088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.205138 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.205218 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:48.205871 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.206414 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.207578 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.207817 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.207906 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.209217 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:48.209220 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:48.209222 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:48.209246 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:48.210526 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.211028 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.211031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.211032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.211106 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.211374 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.211377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.211378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.211644 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:48.211724 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.212059 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.212183 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:39:48.231601 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:36237
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:48.348851 12260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:48.349189 12260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:48.349251 12260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:48.353475 12260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:48.353551 12260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:48.353603 12260 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:39:48.353631 12260 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:39:48.353675 12260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:39:48.359261 12260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.241.254:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:48.360432 12260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:48.361558 12260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:48.371688 12269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:48.371717 12271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:48.371716 12266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:48.372167 12260 server_base.cc:1034] running on GCE node
I20260806 18:39:48.372613 12260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:48.373323 12260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:48.374482 12260 hybrid_clock.cc:648] HybridClock initialized: now 1786041588374462 us; error 35 us; skew 500 ppm
Aug 06 18:39:48 dist-test-slave-k5rr krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041588, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:48.377377 12260 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:39:48.378400 12260 webserver.cc:460] Webserver started at http://127.8.241.195:37007/ using document root <none> and password file <none>
I20260806 18:39:48.378974 12260 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:48.379036 12260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:48.379220 12260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:48.381421 12260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "37fdb5e677a94fab8d06cabc6eed56cf"
format_stamp: "Formatted at 2026-08-06 18:39:48 on dist-test-slave-k5rr"
server_key: "8e30cfe9255d40ef2b4578bc29db7dd6"
server_key_iv: "0e94458d77a4cd3247049e27b3c56f85"
server_key_version: "encryptionkey@0"
I20260806 18:39:48.381968 12260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "37fdb5e677a94fab8d06cabc6eed56cf"
format_stamp: "Formatted at 2026-08-06 18:39:48 on dist-test-slave-k5rr"
server_key: "8e30cfe9255d40ef2b4578bc29db7dd6"
server_key_iv: "0e94458d77a4cd3247049e27b3c56f85"
server_key_version: "encryptionkey@0"
I20260806 18:39:48.385928 12260 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:39:48.388553 12280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:48.389947 12260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:39:48.390064 12260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "37fdb5e677a94fab8d06cabc6eed56cf"
format_stamp: "Formatted at 2026-08-06 18:39:48 on dist-test-slave-k5rr"
server_key: "8e30cfe9255d40ef2b4578bc29db7dd6"
server_key_iv: "0e94458d77a4cd3247049e27b3c56f85"
server_key_version: "encryptionkey@0"
I20260806 18:39:48.390161 12260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:48.414072 12260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:48.417558 12260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:48.417732 12260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:48.419509 12260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:48.419569 12260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:48.419625 12260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:48.419654 12260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:39:48 dist-test-slave-k5rr krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041588, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:48.429998 12260 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:41617
I20260806 18:39:48.430073 12456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:41617 every 8 connection(s)
I20260806 18:39:48.431103 12260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:39:48.434135 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.420290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:41567 (local address 127.8.241.254:36237)
0806 18:39:48.420408 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.420412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.421601 (+ 1189us) server_negotiation.cc:407] Connection header received
0806 18:39:48.422705 (+ 1104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.422709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.422775 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.422834 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:48.424651 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.425537 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.427127 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.427457 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.427576 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.430352 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:48.430356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:48.430359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:48.430388 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:48.431962 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.432513 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.432517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.432519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.432610 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.433032 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.433036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.433038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.433235 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:48.433333 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.433819 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.433968 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:39:48.434942 12290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.420876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36237 (local address 127.8.241.195:41567)
0806 18:39:48.421432 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.421481 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.422443 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.423304 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.423315 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.423757 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:48.424482 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.424506 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.425680 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.425684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.426996 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.427010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.427619 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.427622 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.427773 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.428453 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:48.428471 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:48.430158 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:48.432073 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.432098 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.432114 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.432417 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.432697 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.432700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.432702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.432903 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.433339 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:48.433346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:48.433594 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.433762 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.434107 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":147,"threads_started":1}
I20260806 18:39:48.436116 12458 heartbeater.cc:347] Connected to a master server at 127.8.241.254:36237
I20260806 18:39:48.436272 12458 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:48.436899 12458 heartbeater.cc:511] Master 127.8.241.254:36237 requested a full tablet report, sending...
I20260806 18:39:48.438166 11706 ts_manager.cc:194] Registered new tserver with Master: 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617)
I20260806 18:39:48.438844 11706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:41567
I20260806 18:39:48.439183 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 12260
I20260806 18:39:48.439307 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:39:48.439620 9159 external_mini_cluster.cc:1428] Setting key a41ae5c30f776ac5016f529603f157fc
I20260806 18:39:48.441665 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:39:48.445883 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.438647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:48105 (local address 127.8.241.254:36237)
0806 18:39:48.438775 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.438780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.439022 (+ 242us) server_negotiation.cc:407] Connection header received
0806 18:39:48.439175 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.439179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.439227 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.439278 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:48.439914 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.440555 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.441586 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.441842 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.441930 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.443032 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:48.443036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:48.443039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:48.443064 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:48.444349 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.444739 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.444743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.444745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.444819 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.445093 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.445097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.445099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.445285 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:48.445374 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.445628 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.445735 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:39:48.445895 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.438570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36237 (local address 127.8.241.195:48105)
0806 18:39:48.438939 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.438962 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.439064 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.439421 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.439424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.439651 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:48.439773 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.439781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.440704 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.440707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.441472 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.441482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.441942 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.441954 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.442056 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.442538 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:48.442549 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:48.442889 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:48.444454 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.444457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.444459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.444651 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.444901 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.444904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.444906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.444995 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.445376 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:48.445380 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:48.445475 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.445560 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.445709 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":83,"threads_started":1}
Aug 06 18:39:48 dist-test-slave-k5rr krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:39:48 dist-test-slave-k5rr krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:48.467480 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.460073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.8.241.254:36237)
0806 18:39:48.460195 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.460198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.460211 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:39:48.460255 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.460258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.460299 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.460368 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:48.460991 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.461539 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.462401 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.462609 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.462683 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.464887 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:48.464891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:48.464893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:48.464918 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:48.466123 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.466519 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.466522 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.466523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.466597 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.466779 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.466781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.466782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.466945 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:48.467058 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.467225 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.467319 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:39:48.470386 11706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:39:48.473429 11706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:39:48.488312 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.481764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:35197 (local address 127.0.0.1:39366)
0806 18:39:48.482232 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.482287 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.482420 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.483514 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.483520 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.483543 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:48.483698 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.483706 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.485350 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.485353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.486735 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.486747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.487750 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.487754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.487865 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.487905 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.488017 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.488151 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":16,"thread_start_us":68,"threads_started":1}
I20260806 18:39:48.488888 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.481892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39366 (local address 127.8.241.194:35197)
0806 18:39:48.482891 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.482903 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.482927 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:39:48.483010 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.483016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.483242 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.483844 (+ 602us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:48.483971 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.485202 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.486875 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.487630 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.487725 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.487886 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.488625 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.488731 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"thread_start_us":142,"threads_started":1}
I20260806 18:39:48.488905 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.481841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:41617 (local address 127.0.0.1:40002)
0806 18:39:48.483047 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.483060 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.483144 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.483786 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.483789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.483807 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:48.483907 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.483913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.485336 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.485340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.487261 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.487272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.488301 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.488305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.488401 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.488626 (+ 225us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.488700 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.488775 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1150,"mutex_wait_us":33,"thread_start_us":80,"threads_started":1}
I20260806 18:39:48.489336 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.482620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44857 (local address 127.0.0.1:55762)
0806 18:39:48.483421 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.483436 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.483685 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.484654 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.484656 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.484700 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:48.484869 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.484878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.486802 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.486805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.487905 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.487916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.488915 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.488919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.489059 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.489084 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.489150 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.489219 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":96,"threads_started":1}
I20260806 18:39:48.489570 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.482893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55762 (local address 127.8.241.193:44857)
0806 18:39:48.483254 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.483260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.483578 (+ 318us) server_negotiation.cc:407] Connection header received
0806 18:39:48.483901 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.483934 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.484206 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.484381 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:48.485071 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.486642 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.488046 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.488784 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.488891 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.489080 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.489238 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.489335 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":89,"threads_started":1}
I20260806 18:39:48.489578 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.482508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40002 (local address 127.8.241.195:41617)
0806 18:39:48.482829 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.482842 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.483235 (+ 393us) server_negotiation.cc:407] Connection header received
0806 18:39:48.483317 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.483322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.483507 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.483649 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:48.484048 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.485211 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.487396 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.488187 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.488284 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.488433 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.489203 (+ 770us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.489301 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":111,"threads_started":1}
I20260806 18:39:48.492269 12351 tablet_service.cc:1467] Processing CreateTablet for tablet 5442e04687e04d55bdb3ca15e962f631 (DEFAULT_TABLE table=test-table [id=95e20883255245a9b8379fa6f07f2372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:48.492317 12138 tablet_service.cc:1467] Processing CreateTablet for tablet 5442e04687e04d55bdb3ca15e962f631 (DEFAULT_TABLE table=test-table [id=95e20883255245a9b8379fa6f07f2372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:48.492460 11926 tablet_service.cc:1467] Processing CreateTablet for tablet 5442e04687e04d55bdb3ca15e962f631 (DEFAULT_TABLE table=test-table [id=95e20883255245a9b8379fa6f07f2372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:48.493337 12351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5442e04687e04d55bdb3ca15e962f631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:48.493386 12138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5442e04687e04d55bdb3ca15e962f631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:48.493386 11926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5442e04687e04d55bdb3ca15e962f631. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:48.500567 12490 tablet_bootstrap.cc:492] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: Bootstrap starting.
I20260806 18:39:48.501332 12492 tablet_bootstrap.cc:492] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: Bootstrap starting.
I20260806 18:39:48.502919 12490 tablet_bootstrap.cc:654] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:48.503171 12492 tablet_bootstrap.cc:654] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:48.503669 12490 log.cc:824] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:48.503827 12492 log.cc:824] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:48.504393 12493 tablet_bootstrap.cc:492] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: Bootstrap starting.
I20260806 18:39:48.505677 12492 tablet_bootstrap.cc:492] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: No bootstrap required, opened a new log
I20260806 18:39:48.505868 12490 tablet_bootstrap.cc:492] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: No bootstrap required, opened a new log
I20260806 18:39:48.505873 12492 ts_tablet_manager.cc:1397] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 18:39:48.506057 12490 ts_tablet_manager.cc:1397] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:39:48.506753 12493 tablet_bootstrap.cc:654] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:48.507558 12493 log.cc:824] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:48.509711 12493 tablet_bootstrap.cc:492] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: No bootstrap required, opened a new log
I20260806 18:39:48.509636 12490 raft_consensus.cc:348] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } }
I20260806 18:39:48.509647 12492 raft_consensus.cc:348] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } }
I20260806 18:39:48.509886 12490 raft_consensus.cc:374] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:48.509907 12493 ts_tablet_manager.cc:1397] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:39:48.509900 12492 raft_consensus.cc:374] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:48.509940 12490 raft_consensus.cc:729] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37fdb5e677a94fab8d06cabc6eed56cf, State: Initialized, Role: FOLLOWER
I20260806 18:39:48.509965 12492 raft_consensus.cc:729] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1ae74e03424b7786debc2b73ba5b45, State: Initialized, Role: FOLLOWER
I20260806 18:39:48.510375 12490 consensus_queue.cc:260] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } }
I20260806 18:39:48.510399 12492 consensus_queue.cc:260] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } }
I20260806 18:39:48.511051 12458 heartbeater.cc:503] Master 127.8.241.254:36237 was elected leader, sending a full tablet report...
I20260806 18:39:48.511471 12492 ts_tablet_manager.cc:1428] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:39:48.511973 12030 heartbeater.cc:503] Master 127.8.241.254:36237 was elected leader, sending a full tablet report...
I20260806 18:39:48.513388 12490 ts_tablet_manager.cc:1428] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:39:48.513790 12493 raft_consensus.cc:348] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } }
I20260806 18:39:48.514024 12493 raft_consensus.cc:374] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:48.514070 12493 raft_consensus.cc:729] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a844e184a27450c9b02ffb3573c705e, State: Initialized, Role: FOLLOWER
I20260806 18:39:48.514506 12493 consensus_queue.cc:260] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } }
I20260806 18:39:48.515146 12246 heartbeater.cc:503] Master 127.8.241.254:36237 was elected leader, sending a full tablet report...
I20260806 18:39:48.515452 12493 ts_tablet_manager.cc:1428] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 18:39:48.682200 12460 tablet.cc:2369] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:39:48.696350 12247 tablet.cc:2369] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:39:48.715601 12031 tablet.cc:2369] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:39:48.784322 12501 raft_consensus.cc:482] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:39:48.784601 12501 raft_consensus.cc:504] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } }
I20260806 18:39:48.785914 12501 leader_election.cc:288] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197), 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617)
I20260806 18:39:48.789978 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.786326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:37427 (local address 127.8.241.194:35197)
0806 18:39:48.786476 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.786481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.786731 (+ 250us) server_negotiation.cc:407] Connection header received
0806 18:39:48.786844 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.786848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.786895 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.786967 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:48.787259 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.787792 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.788969 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.789403 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.789522 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.789581 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.789772 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.789845 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:39:48.789996 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.786314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:37701 (local address 127.8.241.195:41617)
0806 18:39:48.786453 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.786458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.786825 (+ 367us) server_negotiation.cc:407] Connection header received
0806 18:39:48.786933 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.786935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.786985 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.787058 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:48.787310 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.787825 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.788964 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.789399 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.789522 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.789581 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.789797 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.789874 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:39:48.790023 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.786201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:35197 (local address 127.8.241.193:37427)
0806 18:39:48.786625 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.786638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.786752 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.786994 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.786996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.787013 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:48.787147 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.787155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.787909 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.787913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.788837 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.788847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.789529 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.789533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.789644 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.789675 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.789735 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.789808 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":35,"thread_start_us":154,"threads_started":1}
I20260806 18:39:48.790026 12510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.786201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:41617 (local address 127.8.241.193:37701)
0806 18:39:48.786712 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.786726 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.786843 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.787088 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.787090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.787104 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:48.787201 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.787207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.787946 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.787950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.788837 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.788846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.789523 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.789527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.789680 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.789698 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.789771 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.789840 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":119,"thread_start_us":120,"threads_started":1}
I20260806 18:39:48.790815 12167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5442e04687e04d55bdb3ca15e962f631" candidate_uuid: "1f1ae74e03424b7786debc2b73ba5b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a844e184a27450c9b02ffb3573c705e" is_pre_election: true
I20260806 18:39:48.790820 12384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5442e04687e04d55bdb3ca15e962f631" candidate_uuid: "1f1ae74e03424b7786debc2b73ba5b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" is_pre_election: true
I20260806 18:39:48.791128 12384 raft_consensus.cc:2454] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1ae74e03424b7786debc2b73ba5b45 in term 0.
I20260806 18:39:48.791131 12167 raft_consensus.cc:2454] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1ae74e03424b7786debc2b73ba5b45 in term 0.
I20260806 18:39:48.791615 11847 leader_election.cc:302] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1ae74e03424b7786debc2b73ba5b45, 37fdb5e677a94fab8d06cabc6eed56cf; no voters:
I20260806 18:39:48.791899 12501 raft_consensus.cc:2781] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:39:48.791998 12501 raft_consensus.cc:482] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:39:48.792049 12501 raft_consensus.cc:3037] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:48.793140 12501 raft_consensus.cc:504] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } }
I20260806 18:39:48.793538 12501 leader_election.cc:288] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [CANDIDATE]: Term 1 election: Requested vote from peers 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197), 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617)
I20260806 18:39:48.793870 12167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5442e04687e04d55bdb3ca15e962f631" candidate_uuid: "1f1ae74e03424b7786debc2b73ba5b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a844e184a27450c9b02ffb3573c705e"
I20260806 18:39:48.794000 12384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5442e04687e04d55bdb3ca15e962f631" candidate_uuid: "1f1ae74e03424b7786debc2b73ba5b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fdb5e677a94fab8d06cabc6eed56cf"
I20260806 18:39:48.794039 12167 raft_consensus.cc:3037] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:48.794137 12384 raft_consensus.cc:3037] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:48.795208 12167 raft_consensus.cc:2454] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1ae74e03424b7786debc2b73ba5b45 in term 1.
I20260806 18:39:48.795555 11853 leader_election.cc:302] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1ae74e03424b7786debc2b73ba5b45, 7a844e184a27450c9b02ffb3573c705e; no voters:
I20260806 18:39:48.795665 12384 raft_consensus.cc:2454] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1ae74e03424b7786debc2b73ba5b45 in term 1.
I20260806 18:39:48.795714 12501 raft_consensus.cc:2781] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:48.795940 12501 raft_consensus.cc:686] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 LEADER]: Becoming Leader. State: Replica: 1f1ae74e03424b7786debc2b73ba5b45, State: Running, Role: LEADER
I20260806 18:39:48.796329 12501 consensus_queue.cc:237] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } }
I20260806 18:39:48.799620 11708 catalog_manager.cc:5495] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193). New cstate: current_term: 1 leader_uuid: "1f1ae74e03424b7786debc2b73ba5b45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } health_report { overall_health: HEALTHY } } }
Aug 06 18:39:48 dist-test-slave-k5rr krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041587, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:48.887182 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.878899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36237 (local address 127.0.0.1:40558)
0806 18:39:48.879198 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.879210 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.879331 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.879571 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.879574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.880026 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:48.880128 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.880135 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.880865 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.880869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.881478 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.881488 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.881912 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.881915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.882006 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.882560 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:48.882572 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:48.884082 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:48.885734 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.885746 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.885753 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.886009 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.886231 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:48.886232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:48.886234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:48.886307 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:48.886629 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:48.886636 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:48.886757 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.886848 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.886991 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
I20260806 18:39:48.887187 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.878993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40558 (local address 127.8.241.254:36237)
0806 18:39:48.879126 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.879131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.879281 (+ 150us) server_negotiation.cc:407] Connection header received
0806 18:39:48.879415 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.879418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.879465 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.879531 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:48.880249 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.880765 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.881598 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.881806 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.881893 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.884246 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:48.884249 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:48.884251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:48.884277 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:48.885632 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.886089 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.886092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.886093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.886165 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:48.886368 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:48.886370 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:48.886371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:48.886550 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:48.886622 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.886895 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.887052 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":46}
I20260806 18:39:48.891381 11708 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:40558:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:39:48.892174 11708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:39:48.897353 12138 tablet_service.cc:1467] Processing CreateTablet for tablet 66223686e9814349ada4e32de92b81f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6cbee4100dce49338d94bb8168a6b387]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:39:48.897442 12351 tablet_service.cc:1467] Processing CreateTablet for tablet 66223686e9814349ada4e32de92b81f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6cbee4100dce49338d94bb8168a6b387]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:39:48.897353 11926 tablet_service.cc:1467] Processing CreateTablet for tablet 66223686e9814349ada4e32de92b81f5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6cbee4100dce49338d94bb8168a6b387]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:39:48.898186 12138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66223686e9814349ada4e32de92b81f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:48.898186 11926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66223686e9814349ada4e32de92b81f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:48.898187 12351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66223686e9814349ada4e32de92b81f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:48.901046 12490 tablet_bootstrap.cc:492] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: Bootstrap starting.
I20260806 18:39:48.901157 12493 tablet_bootstrap.cc:492] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: Bootstrap starting.
I20260806 18:39:48.901975 12493 tablet_bootstrap.cc:654] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:48.902022 12490 tablet_bootstrap.cc:654] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:48.902740 12492 tablet_bootstrap.cc:492] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: Bootstrap starting.
I20260806 18:39:48.903445 12490 tablet_bootstrap.cc:492] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: No bootstrap required, opened a new log
I20260806 18:39:48.903476 12493 tablet_bootstrap.cc:492] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: No bootstrap required, opened a new log
I20260806 18:39:48.903533 12490 ts_tablet_manager.cc:1397] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:39:48.903565 12493 ts_tablet_manager.cc:1397] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:39:48.903952 12490 raft_consensus.cc:348] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:48.904042 12492 tablet_bootstrap.cc:654] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:48.904062 12490 raft_consensus.cc:374] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:48.904093 12490 raft_consensus.cc:729] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37fdb5e677a94fab8d06cabc6eed56cf, State: Initialized, Role: FOLLOWER
I20260806 18:39:48.904042 12493 raft_consensus.cc:348] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:48.904152 12493 raft_consensus.cc:374] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:48.904197 12493 raft_consensus.cc:729] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a844e184a27450c9b02ffb3573c705e, State: Initialized, Role: FOLLOWER
I20260806 18:39:48.904219 12490 consensus_queue.cc:260] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:48.904341 12493 consensus_queue.cc:260] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:48.904627 12493 ts_tablet_manager.cc:1428] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:39:48.904659 12506 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:48.904810 12506 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:48.904757 12500 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:48.904891 12500 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:48.904990 12490 ts_tablet_manager.cc:1428] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:39:48.906389 12492 tablet_bootstrap.cc:492] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: No bootstrap required, opened a new log
I20260806 18:39:48.906499 12492 ts_tablet_manager.cc:1397] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260806 18:39:48.906972 12492 raft_consensus.cc:348] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:48.907083 12492 raft_consensus.cc:374] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:48.907114 12492 raft_consensus.cc:729] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1ae74e03424b7786debc2b73ba5b45, State: Initialized, Role: FOLLOWER
I20260806 18:39:48.907248 12492 consensus_queue.cc:260] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:48.907542 12492 ts_tablet_manager.cc:1428] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 18:39:48.908129 12501 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:48.908262 12501 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:48.967048 12506 raft_consensus.cc:482] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:39:48.967214 12506 raft_consensus.cc:504] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:48.968606 12506 leader_election.cc:288] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857), 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617)
I20260806 18:39:48.973024 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.968984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:49109 (local address 127.8.241.193:44857)
0806 18:39:48.969125 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.969130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.969343 (+ 213us) server_negotiation.cc:407] Connection header received
0806 18:39:48.969579 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.969582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.969633 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.969698 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:48.969972 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.970468 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.972062 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.972470 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.972586 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.972647 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.972807 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.972881 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:39:48.973063 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.968857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44857 (local address 127.8.241.194:49109)
0806 18:39:48.969262 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.969276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.969448 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.969720 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.969723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.969740 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:48.969865 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.969870 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.970571 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.970574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.971915 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.971927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.972606 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.972610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.972698 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.972716 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.972784 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.972855 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":25,"spinlock_wait_cycles":52480,"thread_start_us":132,"threads_started":1}
I20260806 18:39:48.973681 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.968857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:41617 (local address 127.8.241.194:50705)
0806 18:39:48.969311 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:48.969325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:48.969470 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:48.969861 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:48.969864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:48.969880 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:48.970431 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.970439 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.971164 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.971167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.971932 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:48.971941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.972930 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.972934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:48.973073 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.973385 (+ 312us) client_negotiation.cc:769] Sending connection context
0806 18:39:48.973462 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:39:48.973532 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":170,"spinlock_wait_cycles":2176,"thread_start_us":64,"threads_started":1}
I20260806 18:39:48.973613 11957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66223686e9814349ada4e32de92b81f5" candidate_uuid: "7a844e184a27450c9b02ffb3573c705e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1ae74e03424b7786debc2b73ba5b45" is_pre_election: true
I20260806 18:39:48.973711 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:48.969278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:50705 (local address 127.8.241.195:41617)
0806 18:39:48.969395 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:48.969399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:48.969414 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:39:48.969642 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:48.969645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:48.969695 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:48.969759 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:48.970570 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.971052 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.972092 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:48.972715 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:48.972823 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:48.972872 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:48.973500 (+ 628us) server_negotiation.cc:299] Negotiation successful
0806 18:39:48.973594 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:39:48.973875 11957 raft_consensus.cc:2454] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a844e184a27450c9b02ffb3573c705e in term 0.
I20260806 18:39:48.974375 12073 leader_election.cc:302] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1ae74e03424b7786debc2b73ba5b45, 7a844e184a27450c9b02ffb3573c705e; no voters:
I20260806 18:39:48.974437 12384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66223686e9814349ada4e32de92b81f5" candidate_uuid: "7a844e184a27450c9b02ffb3573c705e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" is_pre_election: true
I20260806 18:39:48.974578 12384 raft_consensus.cc:2454] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a844e184a27450c9b02ffb3573c705e in term 0.
I20260806 18:39:48.974681 12506 raft_consensus.cc:2781] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:39:48.974766 12506 raft_consensus.cc:482] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:39:48.974817 12506 raft_consensus.cc:3037] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:48.975821 12506 raft_consensus.cc:504] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:48.976192 12506 leader_election.cc:288] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [CANDIDATE]: Term 1 election: Requested vote from peers 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857), 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617)
I20260806 18:39:48.976547 11957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66223686e9814349ada4e32de92b81f5" candidate_uuid: "7a844e184a27450c9b02ffb3573c705e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1ae74e03424b7786debc2b73ba5b45"
I20260806 18:39:48.976569 12384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66223686e9814349ada4e32de92b81f5" candidate_uuid: "7a844e184a27450c9b02ffb3573c705e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fdb5e677a94fab8d06cabc6eed56cf"
I20260806 18:39:48.976660 11957 raft_consensus.cc:3037] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:48.976684 12384 raft_consensus.cc:3037] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:48.977654 12384 raft_consensus.cc:2454] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a844e184a27450c9b02ffb3573c705e in term 1.
I20260806 18:39:48.977670 11957 raft_consensus.cc:2454] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a844e184a27450c9b02ffb3573c705e in term 1.
I20260806 18:39:48.977965 12068 leader_election.cc:302] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37fdb5e677a94fab8d06cabc6eed56cf, 7a844e184a27450c9b02ffb3573c705e; no voters:
I20260806 18:39:48.978257 12506 raft_consensus.cc:2781] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:48.978492 12506 raft_consensus.cc:686] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [term 1 LEADER]: Becoming Leader. State: Replica: 7a844e184a27450c9b02ffb3573c705e, State: Running, Role: LEADER
I20260806 18:39:48.978888 12506 consensus_queue.cc:237] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:48.980937 12535 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: TxnStatusTablet state changed. Reason: New leader 7a844e184a27450c9b02ffb3573c705e. Latest consensus state: current_term: 1 leader_uuid: "7a844e184a27450c9b02ffb3573c705e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:48.981089 12535 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:39:48.981374 12538 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:39:48.981582 12538 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:39:48.982458 11708 catalog_manager.cc:5495] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a844e184a27450c9b02ffb3573c705e (127.8.241.194). New cstate: current_term: 1 leader_uuid: "7a844e184a27450c9b02ffb3573c705e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } health_report { overall_health: UNKNOWN } } }
I20260806 18:39:49.008679 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.004655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:35197 (local address 127.0.0.1:39378)
0806 18:39:49.004940 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:49.004963 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:49.005113 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:49.005447 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:49.005450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:49.005458 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:39:49.005570 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.005576 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.006275 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.006279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.007038 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.007050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.007489 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.007493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.007578 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.007635 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.008341 (+ 706us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.008358 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 18:39:49.008423 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:39:49.008517 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260806 18:39:49.008739 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.004766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39378 (local address 127.8.241.194:35197)
0806 18:39:49.004891 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:49.004896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:49.005042 (+ 146us) server_negotiation.cc:407] Connection header received
0806 18:39:49.005264 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:49.005267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:49.005332 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:49.005419 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:49.005674 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.006168 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.007159 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.007371 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.007486 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.007776 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.008242 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.008353 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:49.008576 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 18:39:49.008633 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:39:49.014896 12384 raft_consensus.cc:1261] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Refusing update from remote peer 7a844e184a27450c9b02ffb3573c705e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:49.014895 11957 raft_consensus.cc:1261] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Refusing update from remote peer 7a844e184a27450c9b02ffb3573c705e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:49.015659 12506 consensus_queue.cc:1035] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:49.015846 12535 consensus_queue.cc:1035] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:49.019896 12501 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: TxnStatusTablet state changed. Reason: New leader 7a844e184a27450c9b02ffb3573c705e. Latest consensus state: current_term: 1 leader_uuid: "7a844e184a27450c9b02ffb3573c705e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:49.019903 12500 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: TxnStatusTablet state changed. Reason: New leader 7a844e184a27450c9b02ffb3573c705e. Latest consensus state: current_term: 1 leader_uuid: "7a844e184a27450c9b02ffb3573c705e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:49.020033 12501 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:49.020035 12500 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:49.024015 12535 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a844e184a27450c9b02ffb3573c705e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:49.024142 12535 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:39:49.024469 12535 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a844e184a27450c9b02ffb3573c705e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:49.024562 12535 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 7a844e184a27450c9b02ffb3573c705e: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:39:49.024652 12542 mvcc.cc:205] Tried to move back new op lower bound from 7315626348596752384 to 7315626348460294144. Current Snapshot: MvccSnapshot[applied={T|T < 7315626348596752384}]
I20260806 18:39:49.026279 12501 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a844e184a27450c9b02ffb3573c705e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:49.026404 12501 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 1f1ae74e03424b7786debc2b73ba5b45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:49.027878 12500 tablet_replica.cc:439] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a844e184a27450c9b02ffb3573c705e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } } }
I20260806 18:39:49.027999 12500 tablet_replica.cc:442] T 66223686e9814349ada4e32de92b81f5 P 37fdb5e677a94fab8d06cabc6eed56cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:39:49.035426 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.031622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.8.241.193:44857)
0806 18:39:49.031737 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:49.031741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:49.031934 (+ 193us) server_negotiation.cc:407] Connection header received
0806 18:39:49.032098 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:49.032101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:49.032157 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:49.032242 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:49.032494 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.033095 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.034027 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.034251 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.034335 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.034620 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.035058 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.035161 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:49.035241 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 18:39:49.035294 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:39:49.047482 12563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.043619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:35197 (local address 127.8.241.193:35795)
0806 18:39:49.044015 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:49.044027 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:49.044119 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:49.044350 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:49.044352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:49.044358 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:39:49.044475 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.044480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.045476 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.045479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.046128 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.046138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.046708 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.046711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.046784 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.046833 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.047180 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.047195 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 18:39:49.047261 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:39:49.047336 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":94,"threads_started":1}
I20260806 18:39:49.047489 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.043736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:35795 (local address 127.8.241.194:35197)
0806 18:39:49.043863 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:49.043868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:49.044097 (+ 229us) server_negotiation.cc:407] Connection header received
0806 18:39:49.044208 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:49.044211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:49.044266 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:49.044348 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:49.044572 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.045372 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.046233 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.046602 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.046707 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.046975 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.047075 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.047150 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:49.047294 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 18:39:49.047354 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:39:49.054802 12384 raft_consensus.cc:1261] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Refusing update from remote peer 1f1ae74e03424b7786debc2b73ba5b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:49.055128 12167 raft_consensus.cc:1261] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Refusing update from remote peer 1f1ae74e03424b7786debc2b73ba5b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:49.055696 12501 consensus_queue.cc:1035] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:49.055964 12513 consensus_queue.cc:1035] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:49.060859 12544 mvcc.cc:205] Tried to move back new op lower bound from 7315626348763795456 to 7315626347712458752. Current Snapshot: MvccSnapshot[applied={T|T < 7315626348763795456}]
I20260806 18:39:49.082157 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.078425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:59343 (local address 127.8.241.193:44857)
0806 18:39:49.078556 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:49.078561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:49.078823 (+ 262us) server_negotiation.cc:407] Connection header received
0806 18:39:49.078974 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:49.078978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:49.079037 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:49.079111 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:49.079490 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.080191 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.081109 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.081331 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.081427 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.081701 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.081780 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.081837 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:49.081935 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:39:49.081994 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:39:49.082157 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.078328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44857 (local address 127.8.241.194:59343)
0806 18:39:49.078731 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:49.078744 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:49.078848 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:49.079230 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:49.079233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:49.079243 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:39:49.079362 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.079368 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.080314 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.080317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.080966 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.080976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.081429 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.081432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.081521 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.081573 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.081853 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.081859 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:39:49.081925 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:39:49.082010 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":99,"threads_started":1}
I20260806 18:39:49.113809 11708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:49.114049 11708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:49.117496 11708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff: Sending DeleteTablet for 3 replicas of tablet 5442e04687e04d55bdb3ca15e962f631
I20260806 18:39:49.118597 12351 tablet_service.cc:1514] Processing DeleteTablet for tablet 5442e04687e04d55bdb3ca15e962f631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:49 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:40002
I20260806 18:39:49.118614 12138 tablet_service.cc:1514] Processing DeleteTablet for tablet 5442e04687e04d55bdb3ca15e962f631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:49 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:39366
I20260806 18:39:49.119199 12576 tablet_replica.cc:330] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: stopping tablet replica
I20260806 18:39:49.119570 12576 raft_consensus.cc:2229] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:49.119602 11926 tablet_service.cc:1514] Processing DeleteTablet for tablet 5442e04687e04d55bdb3ca15e962f631 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:49 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:55762
I20260806 18:39:49.119813 12576 raft_consensus.cc:2258] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:49.121117 11708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:39:49.121347 12576 ts_tablet_manager.cc:1905] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:39:49.121614 11708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:39:49.123312 12579 tablet_replica.cc:330] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: stopping tablet replica
I20260806 18:39:49.123612 12580 tablet_replica.cc:330] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: stopping tablet replica
I20260806 18:39:49.124006 12580 raft_consensus.cc:2229] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:39:49.124449 12579 raft_consensus.cc:2229] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:49.125373 12580 raft_consensus.cc:2258] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:49.125483 12579 raft_consensus.cc:2258] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:49.127045 12579 ts_tablet_manager.cc:1905] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:49.128508 12580 ts_tablet_manager.cc:1905] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:49.128944 12351 tablet_service.cc:1467] Processing CreateTablet for tablet b26356e9f0364caca6e73fff85c9c40f (DEFAULT_TABLE table=test-table [id=b01aa4511e984122848b96601095fafa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:49.129274 12351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b26356e9f0364caca6e73fff85c9c40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:49.131863 12490 tablet_bootstrap.cc:492] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: Bootstrap starting.
I20260806 18:39:49.132848 12490 tablet_bootstrap.cc:654] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:49.134514 12138 tablet_service.cc:1467] Processing CreateTablet for tablet b26356e9f0364caca6e73fff85c9c40f (DEFAULT_TABLE table=test-table [id=b01aa4511e984122848b96601095fafa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:49.134765 12138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b26356e9f0364caca6e73fff85c9c40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:49.137140 11926 tablet_service.cc:1467] Processing CreateTablet for tablet b26356e9f0364caca6e73fff85c9c40f (DEFAULT_TABLE table=test-table [id=b01aa4511e984122848b96601095fafa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:49.137411 11926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b26356e9f0364caca6e73fff85c9c40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:49.137809 12576 ts_tablet_manager.cc:1918] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:39:49.137912 12576 log.cc:1196] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/5442e04687e04d55bdb3ca15e962f631
I20260806 18:39:49.138005 12490 tablet_bootstrap.cc:492] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: No bootstrap required, opened a new log
I20260806 18:39:49.138083 12490 ts_tablet_manager.cc:1397] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260806 18:39:49.138240 12576 ts_tablet_manager.cc:1939] T 5442e04687e04d55bdb3ca15e962f631 P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting consensus metadata
I20260806 18:39:49.138581 12490 raft_consensus.cc:348] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:49.138697 12490 raft_consensus.cc:374] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:49.138725 12490 raft_consensus.cc:729] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37fdb5e677a94fab8d06cabc6eed56cf, State: Initialized, Role: FOLLOWER
I20260806 18:39:49.138844 12490 consensus_queue.cc:260] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:49.139124 12490 ts_tablet_manager.cc:1428] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260806 18:39:49.139636 11690 catalog_manager.cc:4841] TS 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617): tablet 5442e04687e04d55bdb3ca15e962f631 (table test-table [id=95e20883255245a9b8379fa6f07f2372]) successfully deleted
I20260806 18:39:49.139976 12493 tablet_bootstrap.cc:492] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: Bootstrap starting.
I20260806 18:39:49.140951 12493 tablet_bootstrap.cc:654] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:49.141535 12492 tablet_bootstrap.cc:492] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: Bootstrap starting.
I20260806 18:39:49.142367 12493 tablet_bootstrap.cc:492] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: No bootstrap required, opened a new log
I20260806 18:39:49.142454 12493 ts_tablet_manager.cc:1397] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:39:49.142488 12492 tablet_bootstrap.cc:654] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:49.142946 12493 raft_consensus.cc:348] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:49.143069 12493 raft_consensus.cc:374] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:49.143096 12493 raft_consensus.cc:729] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a844e184a27450c9b02ffb3573c705e, State: Initialized, Role: FOLLOWER
I20260806 18:39:49.143217 12493 consensus_queue.cc:260] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:49.143469 12493 ts_tablet_manager.cc:1428] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:39:49.145550 12492 tablet_bootstrap.cc:492] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: No bootstrap required, opened a new log
I20260806 18:39:49.145656 12492 ts_tablet_manager.cc:1397] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260806 18:39:49.146133 12492 raft_consensus.cc:348] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:49.146272 12492 raft_consensus.cc:374] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:49.146310 12492 raft_consensus.cc:729] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1ae74e03424b7786debc2b73ba5b45, State: Initialized, Role: FOLLOWER
I20260806 18:39:49.146436 12492 consensus_queue.cc:260] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:49.147195 12579 ts_tablet_manager.cc:1918] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:39:49.148001 12580 ts_tablet_manager.cc:1918] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:39:49.148176 12579 log.cc:1196] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/5442e04687e04d55bdb3ca15e962f631
I20260806 18:39:49.148535 12492 ts_tablet_manager.cc:1428] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:39:49.149550 12579 ts_tablet_manager.cc:1939] T 5442e04687e04d55bdb3ca15e962f631 P 7a844e184a27450c9b02ffb3573c705e: Deleting consensus metadata
I20260806 18:39:49.149690 12580 log.cc:1196] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/5442e04687e04d55bdb3ca15e962f631
I20260806 18:39:49.150076 12580 ts_tablet_manager.cc:1939] T 5442e04687e04d55bdb3ca15e962f631 P 1f1ae74e03424b7786debc2b73ba5b45: Deleting consensus metadata
I20260806 18:39:49.151707 11692 catalog_manager.cc:4841] TS 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857): tablet 5442e04687e04d55bdb3ca15e962f631 (table test-table [id=95e20883255245a9b8379fa6f07f2372]) successfully deleted
I20260806 18:39:49.152717 11692 catalog_manager.cc:4841] TS 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197): tablet 5442e04687e04d55bdb3ca15e962f631 (table test-table [id=95e20883255245a9b8379fa6f07f2372]) successfully deleted
I20260806 18:39:49.263912 12500 raft_consensus.cc:482] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:39:49.264076 12500 raft_consensus.cc:504] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:49.265298 12500 leader_election.cc:288] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857), 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197)
I20260806 18:39:49.270059 12591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.266072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:35197 (local address 127.8.241.195:39185)
0806 18:39:49.266423 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:49.266436 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:49.266542 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:49.266781 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:49.266784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:49.266800 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:49.266921 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.266927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.267813 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.267817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.268841 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.268852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.269617 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.269621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.269712 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.269739 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:39:49.269810 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:39:49.269882 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":40,"thread_start_us":83,"threads_started":1}
I20260806 18:39:49.270059 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.266174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:39185 (local address 127.8.241.194:35197)
0806 18:39:49.266299 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:49.266303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:49.266519 (+ 216us) server_negotiation.cc:407] Connection header received
0806 18:39:49.266627 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:49.266630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:49.266681 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:49.266756 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:49.267021 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.267723 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.268970 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.269497 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.269617 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.269676 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:49.269844 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 18:39:49.269931 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:39:49.270545 12167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b26356e9f0364caca6e73fff85c9c40f" candidate_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a844e184a27450c9b02ffb3573c705e" is_pre_election: true
I20260806 18:39:49.270711 12167 raft_consensus.cc:2454] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37fdb5e677a94fab8d06cabc6eed56cf in term 0.
I20260806 18:39:49.270849 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.265693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:55955 (local address 127.8.241.193:44857)
0806 18:39:49.265849 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:49.265854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:49.266065 (+ 211us) server_negotiation.cc:407] Connection header received
0806 18:39:49.266304 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:49.266307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:49.266381 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:49.266425 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:39:49.266855 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.267657 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.269296 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.269851 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.269964 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.270015 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:49.270648 (+ 633us) server_negotiation.cc:299] Negotiation successful
0806 18:39:49.270726 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:39:49.271268 12284 leader_election.cc:302] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37fdb5e677a94fab8d06cabc6eed56cf, 7a844e184a27450c9b02ffb3573c705e; no voters:
I20260806 18:39:49.271471 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.265632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44857 (local address 127.8.241.195:55955)
0806 18:39:49.265955 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:49.265971 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:49.266161 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:49.266567 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:49.266571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:49.266591 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:39:49.266723 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.266731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.267794 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.267797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.269168 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.269180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.270096 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.270100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.270523 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.270544 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:39:49.270614 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:39:49.271333 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":42,"thread_start_us":136,"threads_started":1}
I20260806 18:39:49.271700 12500 raft_consensus.cc:2781] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:39:49.271770 12500 raft_consensus.cc:482] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:39:49.271821 12500 raft_consensus.cc:3037] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:49.271890 11957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b26356e9f0364caca6e73fff85c9c40f" candidate_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1ae74e03424b7786debc2b73ba5b45" is_pre_election: true
I20260806 18:39:49.272039 11957 raft_consensus.cc:2454] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37fdb5e677a94fab8d06cabc6eed56cf in term 0.
I20260806 18:39:49.272890 12500 raft_consensus.cc:504] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:49.273255 12500 leader_election.cc:288] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [CANDIDATE]: Term 1 election: Requested vote from peers 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857), 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197)
I20260806 18:39:49.273578 11957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b26356e9f0364caca6e73fff85c9c40f" candidate_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1ae74e03424b7786debc2b73ba5b45"
I20260806 18:39:49.273679 11957 raft_consensus.cc:3037] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:49.273700 12167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b26356e9f0364caca6e73fff85c9c40f" candidate_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a844e184a27450c9b02ffb3573c705e"
I20260806 18:39:49.273833 12167 raft_consensus.cc:3037] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:49.274564 11957 raft_consensus.cc:2454] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37fdb5e677a94fab8d06cabc6eed56cf in term 1.
I20260806 18:39:49.274784 12167 raft_consensus.cc:2454] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37fdb5e677a94fab8d06cabc6eed56cf in term 1.
I20260806 18:39:49.274838 12284 leader_election.cc:302] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1ae74e03424b7786debc2b73ba5b45, 37fdb5e677a94fab8d06cabc6eed56cf; no voters:
I20260806 18:39:49.275012 12500 raft_consensus.cc:2781] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:49.275233 12500 raft_consensus.cc:686] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 LEADER]: Becoming Leader. State: Replica: 37fdb5e677a94fab8d06cabc6eed56cf, State: Running, Role: LEADER
I20260806 18:39:49.275504 12500 consensus_queue.cc:237] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:49.277760 11708 catalog_manager.cc:5495] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195). New cstate: current_term: 1 leader_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } health_report { overall_health: HEALTHY } } }
I20260806 18:39:49.301294 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.298044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.8.241.195:41617)
0806 18:39:49.298155 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:49.298159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:49.298173 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:39:49.298309 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:49.298311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:49.298366 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:49.298461 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:49.298629 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.299134 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.299918 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.300147 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.300232 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.300448 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.300925 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.301085 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:49.301120 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 18:39:49.301172 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:39:49.312292 12486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.309179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:55665 (local address 127.8.241.194:35197)
0806 18:39:49.309321 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:49.309326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:49.309619 (+ 293us) server_negotiation.cc:407] Connection header received
0806 18:39:49.309767 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:49.309769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:49.309818 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:49.309883 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:49.310096 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.310586 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.311378 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.311612 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.311685 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.311921 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.312003 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.312054 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:49.312120 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 18:39:49.312173 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:39:49.312314 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.309076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:35197 (local address 127.8.241.195:55665)
0806 18:39:49.309530 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:49.309546 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:49.309674 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:49.309900 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:49.309903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:49.309910 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:39:49.310020 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.310024 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.310671 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.310673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.311281 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.311288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.311696 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.311699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.311763 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.311812 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.312065 (+ 253us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.312068 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:39:49.312111 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 18:39:49.312175 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":99,"threads_started":1}
I20260806 18:39:49.318166 11957 raft_consensus.cc:1261] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Refusing update from remote peer 37fdb5e677a94fab8d06cabc6eed56cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:49.318214 12167 raft_consensus.cc:1261] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Refusing update from remote peer 37fdb5e677a94fab8d06cabc6eed56cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:49.318810 12593 consensus_queue.cc:1035] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:49.319020 12500 consensus_queue.cc:1035] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:49.322387 12543 mvcc.cc:205] Tried to move back new op lower bound from 7315626349843062784 to 7315626349674795008. Current Snapshot: MvccSnapshot[applied={T|T < 7315626349843062784}]
I20260806 18:39:49.337318 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.334253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:41617 (local address 127.8.241.194:34211)
0806 18:39:49.334381 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:49.334397 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:49.334478 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:49.334706 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:49.334710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:49.334718 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:39:49.334831 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.334837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.335521 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.335524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.336162 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:49.336172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.336571 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.336575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:49.336657 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.336693 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.337044 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.337053 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 18:39:49.337126 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:39:49.337209 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 18:39:49.337332 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:49.334331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:34211 (local address 127.8.241.195:41617)
0806 18:39:49.334462 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:49.334466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:49.334481 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:39:49.334550 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:49.334552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:49.334605 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:49.334672 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:49.334936 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.335435 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.336249 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:49.336473 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:49.336537 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:49.336834 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:49.336933 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:49.337026 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:49.337145 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 18:39:49.337203 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:39:49.338130 12543 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:39:50.900825 11706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:50.901058 11706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:50.903815 11706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff: Sending DeleteTablet for 3 replicas of tablet b26356e9f0364caca6e73fff85c9c40f
I20260806 18:39:50.904480 12351 tablet_service.cc:1514] Processing DeleteTablet for tablet b26356e9f0364caca6e73fff85c9c40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:50 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:40002
I20260806 18:39:50.904510 11926 tablet_service.cc:1514] Processing DeleteTablet for tablet b26356e9f0364caca6e73fff85c9c40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:50 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:55762
I20260806 18:39:50.904693 12138 tablet_service.cc:1514] Processing DeleteTablet for tablet b26356e9f0364caca6e73fff85c9c40f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:50 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:39366
I20260806 18:39:50.904851 12643 tablet_replica.cc:330] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: stopping tablet replica
I20260806 18:39:50.904961 12645 tablet_replica.cc:330] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: stopping tablet replica
I20260806 18:39:50.905063 12643 raft_consensus.cc:2229] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:39:50.905076 12646 tablet_replica.cc:330] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: stopping tablet replica
I20260806 18:39:50.905200 12646 raft_consensus.cc:2229] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:50.905221 12645 raft_consensus.cc:2229] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:50.905333 12646 raft_consensus.cc:2258] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:50.905441 12643 raft_consensus.cc:2258] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:50.905473 12645 raft_consensus.cc:2258] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:50.906065 12646 ts_tablet_manager.cc:1905] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:50.906279 12643 ts_tablet_manager.cc:1905] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:50.906544 12645 ts_tablet_manager.cc:1905] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:50.909147 12643 ts_tablet_manager.cc:1918] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:39:50.909174 12646 ts_tablet_manager.cc:1918] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:39:50.909241 12646 log.cc:1196] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/b26356e9f0364caca6e73fff85c9c40f
I20260806 18:39:50.909245 12643 log.cc:1196] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/b26356e9f0364caca6e73fff85c9c40f
I20260806 18:39:50.909339 12645 ts_tablet_manager.cc:1918] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:39:50.909418 12645 log.cc:1196] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/b26356e9f0364caca6e73fff85c9c40f
I20260806 18:39:50.909518 12646 ts_tablet_manager.cc:1939] T b26356e9f0364caca6e73fff85c9c40f P 7a844e184a27450c9b02ffb3573c705e: Deleting consensus metadata
I20260806 18:39:50.909538 12643 ts_tablet_manager.cc:1939] T b26356e9f0364caca6e73fff85c9c40f P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting consensus metadata
I20260806 18:39:50.909729 12645 ts_tablet_manager.cc:1939] T b26356e9f0364caca6e73fff85c9c40f P 1f1ae74e03424b7786debc2b73ba5b45: Deleting consensus metadata
I20260806 18:39:50.910326 11692 catalog_manager.cc:4841] TS 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857): tablet b26356e9f0364caca6e73fff85c9c40f (table test-table [id=b01aa4511e984122848b96601095fafa]) successfully deleted
I20260806 18:39:50.910351 11690 catalog_manager.cc:4841] TS 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617): tablet b26356e9f0364caca6e73fff85c9c40f (table test-table [id=b01aa4511e984122848b96601095fafa]) successfully deleted
I20260806 18:39:50.910538 11692 catalog_manager.cc:4841] TS 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197): tablet b26356e9f0364caca6e73fff85c9c40f (table test-table [id=b01aa4511e984122848b96601095fafa]) successfully deleted
Aug 06 18:39:50 dist-test-slave-k5rr krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041590, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:39:50 dist-test-slave-k5rr krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041590, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:50.936192 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:50.927494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40560 (local address 127.8.241.254:36237)
0806 18:39:50.927867 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:50.927874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:50.927893 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:39:50.927947 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:50.927950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:50.928000 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:50.928097 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:50.928831 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:50.929537 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:50.930412 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:50.930657 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:50.930755 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:50.933214 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:50.933217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:50.933219 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:50.933247 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:50.934531 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:50.934918 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:50.934922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:50.934923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:50.935007 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:50.935267 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:50.935271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:50.935272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:50.935488 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:50.935638 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:50.935879 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 18:39:50.936017 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":138,"threads_started":1}
I20260806 18:39:50.938813 11706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:39:50.939255 11706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:39:50.943954 12138 tablet_service.cc:1467] Processing CreateTablet for tablet 392a516be1dd47e7a9e5f8967a15145c (DEFAULT_TABLE table=test-table [id=61a5e484395247519b8bd70f14cb4001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:50.944065 12351 tablet_service.cc:1467] Processing CreateTablet for tablet 392a516be1dd47e7a9e5f8967a15145c (DEFAULT_TABLE table=test-table [id=61a5e484395247519b8bd70f14cb4001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:50.944118 11926 tablet_service.cc:1467] Processing CreateTablet for tablet 392a516be1dd47e7a9e5f8967a15145c (DEFAULT_TABLE table=test-table [id=61a5e484395247519b8bd70f14cb4001]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:50.944291 12138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 392a516be1dd47e7a9e5f8967a15145c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:50.944300 12351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 392a516be1dd47e7a9e5f8967a15145c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:50.944434 11926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 392a516be1dd47e7a9e5f8967a15145c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:50.947039 12664 tablet_bootstrap.cc:492] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: Bootstrap starting.
I20260806 18:39:50.947218 12658 tablet_bootstrap.cc:492] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: Bootstrap starting.
I20260806 18:39:50.947896 12664 tablet_bootstrap.cc:654] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:50.948076 12658 tablet_bootstrap.cc:654] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:50.948535 12659 tablet_bootstrap.cc:492] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: Bootstrap starting.
I20260806 18:39:50.949271 12658 tablet_bootstrap.cc:492] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: No bootstrap required, opened a new log
I20260806 18:39:50.949363 12658 ts_tablet_manager.cc:1397] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:39:50.949492 12659 tablet_bootstrap.cc:654] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:50.949908 12664 tablet_bootstrap.cc:492] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: No bootstrap required, opened a new log
I20260806 18:39:50.949844 12658 raft_consensus.cc:348] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:50.949980 12658 raft_consensus.cc:374] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:50.950014 12658 raft_consensus.cc:729] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1ae74e03424b7786debc2b73ba5b45, State: Initialized, Role: FOLLOWER
I20260806 18:39:50.950022 12664 ts_tablet_manager.cc:1397] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:39:50.950148 12658 consensus_queue.cc:260] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:50.950533 12658 ts_tablet_manager.cc:1428] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:39:50.950525 12664 raft_consensus.cc:348] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:50.950646 12664 raft_consensus.cc:374] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:50.950688 12664 raft_consensus.cc:729] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a844e184a27450c9b02ffb3573c705e, State: Initialized, Role: FOLLOWER
I20260806 18:39:50.950819 12664 consensus_queue.cc:260] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:50.951125 12664 ts_tablet_manager.cc:1428] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:39:50.952050 12659 tablet_bootstrap.cc:492] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: No bootstrap required, opened a new log
I20260806 18:39:50.952154 12659 ts_tablet_manager.cc:1397] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:39:50.952571 12659 raft_consensus.cc:348] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:50.952696 12659 raft_consensus.cc:374] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:50.952728 12659 raft_consensus.cc:729] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37fdb5e677a94fab8d06cabc6eed56cf, State: Initialized, Role: FOLLOWER
I20260806 18:39:50.952860 12659 consensus_queue.cc:260] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:50.953176 12659 ts_tablet_manager.cc:1428] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:39:50.962455 12668 raft_consensus.cc:482] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:39:50.962587 12668 raft_consensus.cc:504] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:50.963068 12668 leader_election.cc:288] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197), 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617)
I20260806 18:39:50.963425 12384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "392a516be1dd47e7a9e5f8967a15145c" candidate_uuid: "1f1ae74e03424b7786debc2b73ba5b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" is_pre_election: true
I20260806 18:39:50.963431 12167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "392a516be1dd47e7a9e5f8967a15145c" candidate_uuid: "1f1ae74e03424b7786debc2b73ba5b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a844e184a27450c9b02ffb3573c705e" is_pre_election: true
I20260806 18:39:50.963570 12384 raft_consensus.cc:2454] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1ae74e03424b7786debc2b73ba5b45 in term 0.
I20260806 18:39:50.963593 12167 raft_consensus.cc:2454] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f1ae74e03424b7786debc2b73ba5b45 in term 0.
I20260806 18:39:50.963872 11847 leader_election.cc:302] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1ae74e03424b7786debc2b73ba5b45, 37fdb5e677a94fab8d06cabc6eed56cf; no voters:
I20260806 18:39:50.964058 12668 raft_consensus.cc:2781] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:39:50.964129 12668 raft_consensus.cc:482] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:39:50.964156 12668 raft_consensus.cc:3037] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:50.965241 12668 raft_consensus.cc:504] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:50.965592 12668 leader_election.cc:288] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [CANDIDATE]: Term 1 election: Requested vote from peers 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197), 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617)
I20260806 18:39:50.965957 12384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "392a516be1dd47e7a9e5f8967a15145c" candidate_uuid: "1f1ae74e03424b7786debc2b73ba5b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fdb5e677a94fab8d06cabc6eed56cf"
I20260806 18:39:50.966080 12384 raft_consensus.cc:3037] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:50.967247 12384 raft_consensus.cc:2454] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1ae74e03424b7786debc2b73ba5b45 in term 1.
I20260806 18:39:50.967548 11847 leader_election.cc:302] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f1ae74e03424b7786debc2b73ba5b45, 37fdb5e677a94fab8d06cabc6eed56cf; no voters:
I20260806 18:39:50.967669 12167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "392a516be1dd47e7a9e5f8967a15145c" candidate_uuid: "1f1ae74e03424b7786debc2b73ba5b45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a844e184a27450c9b02ffb3573c705e"
I20260806 18:39:50.967729 12668 raft_consensus.cc:2781] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:50.967796 12167 raft_consensus.cc:3037] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:50.967949 12668 raft_consensus.cc:686] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 LEADER]: Becoming Leader. State: Replica: 1f1ae74e03424b7786debc2b73ba5b45, State: Running, Role: LEADER
I20260806 18:39:50.968111 12668 consensus_queue.cc:237] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:50.968955 12167 raft_consensus.cc:2454] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f1ae74e03424b7786debc2b73ba5b45 in term 1.
I20260806 18:39:50.969964 11706 catalog_manager.cc:5495] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193). New cstate: current_term: 1 leader_uuid: "1f1ae74e03424b7786debc2b73ba5b45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } health_report { overall_health: UNKNOWN } } }
I20260806 18:39:50.987967 12676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:50.985040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55774 (local address 127.8.241.193:44857)
0806 18:39:50.985293 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:50.985300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:50.985312 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:39:50.985353 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:50.985356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:50.985396 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:50.985465 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:50.985662 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:50.986191 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:50.987116 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:50.987353 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:50.987431 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:50.987623 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:50.987692 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:50.987737 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:50.987801 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 18:39:50.987847 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":99,"threads_started":1}
I20260806 18:39:50.994316 12384 raft_consensus.cc:1261] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Refusing update from remote peer 1f1ae74e03424b7786debc2b73ba5b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:50.994366 12167 raft_consensus.cc:1261] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Refusing update from remote peer 1f1ae74e03424b7786debc2b73ba5b45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:50.994709 12668 consensus_queue.cc:1035] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:50.994875 12669 consensus_queue.cc:1035] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:51.026858 11705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:51.027034 11705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:51.029137 11705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff: Sending DeleteTablet for 3 replicas of tablet 392a516be1dd47e7a9e5f8967a15145c
I20260806 18:39:51.029822 12351 tablet_service.cc:1514] Processing DeleteTablet for tablet 392a516be1dd47e7a9e5f8967a15145c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:51 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:40002
I20260806 18:39:51.029803 11926 tablet_service.cc:1514] Processing DeleteTablet for tablet 392a516be1dd47e7a9e5f8967a15145c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:51 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:55762
I20260806 18:39:51.029842 12138 tablet_service.cc:1514] Processing DeleteTablet for tablet 392a516be1dd47e7a9e5f8967a15145c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:51 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:39366
I20260806 18:39:51.030031 12645 tablet_replica.cc:330] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: stopping tablet replica
I20260806 18:39:51.030031 12643 tablet_replica.cc:330] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: stopping tablet replica
I20260806 18:39:51.030052 12646 tablet_replica.cc:330] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: stopping tablet replica
I20260806 18:39:51.030161 12645 raft_consensus.cc:2229] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:39:51.030161 12643 raft_consensus.cc:2229] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:51.030171 12646 raft_consensus.cc:2229] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:51.030297 12646 raft_consensus.cc:2258] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:51.030299 12643 raft_consensus.cc:2258] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:51.030393 12645 raft_consensus.cc:2258] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:51.031026 12646 ts_tablet_manager.cc:1905] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:51.031221 12643 ts_tablet_manager.cc:1905] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:51.031330 12645 ts_tablet_manager.cc:1905] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:51.031280 11705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:39:51.031782 11705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:39:51.033783 12646 ts_tablet_manager.cc:1918] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:39:51.033783 12643 ts_tablet_manager.cc:1918] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:39:51.033860 12646 log.cc:1196] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/392a516be1dd47e7a9e5f8967a15145c
I20260806 18:39:51.033866 12643 log.cc:1196] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/392a516be1dd47e7a9e5f8967a15145c
I20260806 18:39:51.034116 12643 ts_tablet_manager.cc:1939] T 392a516be1dd47e7a9e5f8967a15145c P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting consensus metadata
I20260806 18:39:51.034116 12646 ts_tablet_manager.cc:1939] T 392a516be1dd47e7a9e5f8967a15145c P 7a844e184a27450c9b02ffb3573c705e: Deleting consensus metadata
I20260806 18:39:51.034132 12645 ts_tablet_manager.cc:1918] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:39:51.034196 12645 log.cc:1196] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/392a516be1dd47e7a9e5f8967a15145c
I20260806 18:39:51.034494 12645 ts_tablet_manager.cc:1939] T 392a516be1dd47e7a9e5f8967a15145c P 1f1ae74e03424b7786debc2b73ba5b45: Deleting consensus metadata
I20260806 18:39:51.034868 11690 catalog_manager.cc:4841] TS 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617): tablet 392a516be1dd47e7a9e5f8967a15145c (table test-table [id=61a5e484395247519b8bd70f14cb4001]) successfully deleted
I20260806 18:39:51.035185 11692 catalog_manager.cc:4841] TS 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857): tablet 392a516be1dd47e7a9e5f8967a15145c (table test-table [id=61a5e484395247519b8bd70f14cb4001]) successfully deleted
I20260806 18:39:51.035370 11692 catalog_manager.cc:4841] TS 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197): tablet 392a516be1dd47e7a9e5f8967a15145c (table test-table [id=61a5e484395247519b8bd70f14cb4001]) successfully deleted
I20260806 18:39:51.038760 11926 tablet_service.cc:1467] Processing CreateTablet for tablet b60c3931203e4834ae6917e1e8de2521 (DEFAULT_TABLE table=test-table [id=696882c8c5344ace8b350c608349e702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:51.038762 12138 tablet_service.cc:1467] Processing CreateTablet for tablet b60c3931203e4834ae6917e1e8de2521 (DEFAULT_TABLE table=test-table [id=696882c8c5344ace8b350c608349e702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:51.038868 12351 tablet_service.cc:1467] Processing CreateTablet for tablet b60c3931203e4834ae6917e1e8de2521 (DEFAULT_TABLE table=test-table [id=696882c8c5344ace8b350c608349e702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:39:51.039049 11926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60c3931203e4834ae6917e1e8de2521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:51.039093 12138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60c3931203e4834ae6917e1e8de2521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:51.039093 12351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60c3931203e4834ae6917e1e8de2521. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:51.040979 12658 tablet_bootstrap.cc:492] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45: Bootstrap starting.
I20260806 18:39:51.041314 12664 tablet_bootstrap.cc:492] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: Bootstrap starting.
I20260806 18:39:51.041392 12659 tablet_bootstrap.cc:492] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: Bootstrap starting.
I20260806 18:39:51.041837 12658 tablet_bootstrap.cc:654] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:51.042162 12659 tablet_bootstrap.cc:654] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:51.042162 12664 tablet_bootstrap.cc:654] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:51.043285 12659 tablet_bootstrap.cc:492] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: No bootstrap required, opened a new log
I20260806 18:39:51.043382 12659 ts_tablet_manager.cc:1397] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:39:51.043566 12664 tablet_bootstrap.cc:492] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: No bootstrap required, opened a new log
I20260806 18:39:51.043658 12664 ts_tablet_manager.cc:1397] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:39:51.043826 12659 raft_consensus.cc:348] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:51.043956 12659 raft_consensus.cc:374] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:51.043993 12659 raft_consensus.cc:729] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37fdb5e677a94fab8d06cabc6eed56cf, State: Initialized, Role: FOLLOWER
I20260806 18:39:51.044023 12664 raft_consensus.cc:348] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:51.044130 12664 raft_consensus.cc:374] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:51.044168 12664 raft_consensus.cc:729] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a844e184a27450c9b02ffb3573c705e, State: Initialized, Role: FOLLOWER
I20260806 18:39:51.044130 12659 consensus_queue.cc:260] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:51.044289 12664 consensus_queue.cc:260] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:51.044438 12659 ts_tablet_manager.cc:1428] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:39:51.044540 12664 ts_tablet_manager.cc:1428] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:39:51.045692 12658 tablet_bootstrap.cc:492] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45: No bootstrap required, opened a new log
I20260806 18:39:51.045799 12658 ts_tablet_manager.cc:1397] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 18:39:51.046248 12658 raft_consensus.cc:348] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:51.046370 12658 raft_consensus.cc:374] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:51.046412 12658 raft_consensus.cc:729] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1ae74e03424b7786debc2b73ba5b45, State: Initialized, Role: FOLLOWER
I20260806 18:39:51.046546 12658 consensus_queue.cc:260] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:51.046844 12658 ts_tablet_manager.cc:1428] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:39:51.095829 12506 raft_consensus.cc:482] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:39:51.095983 12506 raft_consensus.cc:504] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:51.096370 12506 leader_election.cc:288] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857), 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617)
I20260806 18:39:51.096819 11957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60c3931203e4834ae6917e1e8de2521" candidate_uuid: "7a844e184a27450c9b02ffb3573c705e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1ae74e03424b7786debc2b73ba5b45" is_pre_election: true
I20260806 18:39:51.096820 12384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60c3931203e4834ae6917e1e8de2521" candidate_uuid: "7a844e184a27450c9b02ffb3573c705e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" is_pre_election: true
I20260806 18:39:51.096976 11957 raft_consensus.cc:2454] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a844e184a27450c9b02ffb3573c705e in term 0.
I20260806 18:39:51.096987 12384 raft_consensus.cc:2454] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a844e184a27450c9b02ffb3573c705e in term 0.
I20260806 18:39:51.097309 12068 leader_election.cc:302] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37fdb5e677a94fab8d06cabc6eed56cf, 7a844e184a27450c9b02ffb3573c705e; no voters:
I20260806 18:39:51.097489 12506 raft_consensus.cc:2781] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:39:51.097563 12506 raft_consensus.cc:482] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:39:51.097599 12506 raft_consensus.cc:3037] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:51.098574 12506 raft_consensus.cc:504] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:51.098901 12506 leader_election.cc:288] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [CANDIDATE]: Term 1 election: Requested vote from peers 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857), 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617)
I20260806 18:39:51.099298 11957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60c3931203e4834ae6917e1e8de2521" candidate_uuid: "7a844e184a27450c9b02ffb3573c705e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f1ae74e03424b7786debc2b73ba5b45"
I20260806 18:39:51.099298 12384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60c3931203e4834ae6917e1e8de2521" candidate_uuid: "7a844e184a27450c9b02ffb3573c705e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fdb5e677a94fab8d06cabc6eed56cf"
I20260806 18:39:51.099429 11957 raft_consensus.cc:3037] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:51.099429 12384 raft_consensus.cc:3037] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:51.100649 12384 raft_consensus.cc:2454] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a844e184a27450c9b02ffb3573c705e in term 1.
I20260806 18:39:51.100649 11957 raft_consensus.cc:2454] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a844e184a27450c9b02ffb3573c705e in term 1.
I20260806 18:39:51.100971 12073 leader_election.cc:302] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f1ae74e03424b7786debc2b73ba5b45, 37fdb5e677a94fab8d06cabc6eed56cf, 7a844e184a27450c9b02ffb3573c705e; no voters:
I20260806 18:39:51.101174 12506 raft_consensus.cc:2781] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:51.101269 12506 raft_consensus.cc:686] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 1 LEADER]: Becoming Leader. State: Replica: 7a844e184a27450c9b02ffb3573c705e, State: Running, Role: LEADER
I20260806 18:39:51.101419 12506 consensus_queue.cc:237] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } }
I20260806 18:39:51.103084 11705 catalog_manager.cc:5495] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a844e184a27450c9b02ffb3573c705e (127.8.241.194). New cstate: current_term: 1 leader_uuid: "7a844e184a27450c9b02ffb3573c705e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a844e184a27450c9b02ffb3573c705e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 } health_report { overall_health: UNKNOWN } } }
I20260806 18:39:51.130285 12689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:51.127453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39386 (local address 127.8.241.194:35197)
0806 18:39:51.127758 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:51.127765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:51.127777 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:39:51.127817 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:51.127819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:51.127861 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:51.127932 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:51.128140 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:51.128658 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:51.129430 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:51.129651 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:51.129721 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:51.129888 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:51.129958 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:51.130004 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:51.130079 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:39:51.130123 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":145,"threads_started":1}
I20260806 18:39:51.139180 12689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:51.136403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:43953 (local address 127.8.241.194:35197)
0806 18:39:51.136527 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:51.136531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:51.136604 (+ 73us) server_negotiation.cc:407] Connection header received
0806 18:39:51.136707 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:51.136709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:51.136749 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:51.136810 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:39:51.136998 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:51.137577 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:51.138351 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:51.138561 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:51.138627 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:51.138811 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:39:51.138878 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:39:51.138924 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:51.139000 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 18:39:51.139047 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":53}
I20260806 18:39:51.139191 12695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:51.136261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:35197 (local address 127.8.241.194:43953)
0806 18:39:51.136533 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:51.136545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:51.136634 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:51.136824 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:51.136826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:51.136832 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:39:51.136917 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:51.136921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:51.137661 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:51.137663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:51.138264 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:51.138272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:51.138633 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:51.138636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:51.138707 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:51.138733 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:39:51.138931 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:39:51.138934 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:39:51.138991 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 18:39:51.139051 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260806 18:39:51.144475 11957 raft_consensus.cc:1261] T b60c3931203e4834ae6917e1e8de2521 P 1f1ae74e03424b7786debc2b73ba5b45 [term 1 FOLLOWER]: Refusing update from remote peer 7a844e184a27450c9b02ffb3573c705e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:51.144474 12384 raft_consensus.cc:1261] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Refusing update from remote peer 7a844e184a27450c9b02ffb3573c705e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:39:51.144850 12506 consensus_queue.cc:1035] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37fdb5e677a94fab8d06cabc6eed56cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 41617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:39:51.145079 12609 consensus_queue.cc:1035] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f1ae74e03424b7786debc2b73ba5b45" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:39:51.160236 12606 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:39:52.714965 11706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:52.715130 11706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:39:52.717749 11706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7768074f117e497d821a767accea3aff: Sending DeleteTablet for 3 replicas of tablet b60c3931203e4834ae6917e1e8de2521
I20260806 18:39:52.718405 12135 tablet_service.cc:1514] Processing DeleteTablet for tablet b60c3931203e4834ae6917e1e8de2521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:52 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:39366
I20260806 18:39:52.718369 12351 tablet_service.cc:1514] Processing DeleteTablet for tablet b60c3931203e4834ae6917e1e8de2521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:52 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:40002
I20260806 18:39:52.718755 11926 tablet_service.cc:1514] Processing DeleteTablet for tablet b60c3931203e4834ae6917e1e8de2521 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:39:52 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:55762
I20260806 18:39:52.718815 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 11823
I20260806 18:39:52.729323 12730 tablet_replica.cc:330] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: stopping tablet replica
I20260806 18:39:52.729518 12730 raft_consensus.cc:2229] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:39:52.729712 12732 tablet_replica.cc:330] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: stopping tablet replica
I20260806 18:39:52.729902 12732 raft_consensus.cc:2229] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:39:52.729923 12730 raft_consensus.cc:2258] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:52.730291 12732 raft_consensus.cc:2258] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:39:52.730661 12730 ts_tablet_manager.cc:1905] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:52.730969 12732 ts_tablet_manager.cc:1905] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:39:52.732972 12730 ts_tablet_manager.cc:1918] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:39:52.733083 12730 log.cc:1196] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/b60c3931203e4834ae6917e1e8de2521
I20260806 18:39:52.733366 12730 ts_tablet_manager.cc:1939] T b60c3931203e4834ae6917e1e8de2521 P 7a844e184a27450c9b02ffb3573c705e: Deleting consensus metadata
I20260806 18:39:52.733783 12732 ts_tablet_manager.cc:1918] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:39:52.733886 12732 log.cc:1196] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/b60c3931203e4834ae6917e1e8de2521
I20260806 18:39:52.734169 11692 catalog_manager.cc:4841] TS 7a844e184a27450c9b02ffb3573c705e (127.8.241.194:35197): tablet b60c3931203e4834ae6917e1e8de2521 (table test-table [id=696882c8c5344ace8b350c608349e702]) successfully deleted
I20260806 18:39:52.734236 12732 ts_tablet_manager.cc:1939] T b60c3931203e4834ae6917e1e8de2521 P 37fdb5e677a94fab8d06cabc6eed56cf: Deleting consensus metadata
I20260806 18:39:52.734944 11690 catalog_manager.cc:4841] TS 37fdb5e677a94fab8d06cabc6eed56cf (127.8.241.195:41617): tablet b60c3931203e4834ae6917e1e8de2521 (table test-table [id=696882c8c5344ace8b350c608349e702]) successfully deleted
W20260806 18:39:52.735107 11692 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:44857 (error 108)
I20260806 18:39:52.735349 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 12040
I20260806 18:39:52.743772 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:52.743231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44857 (local address 127.0.0.1:48344)
0806 18:39:52.743560 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:52.743662 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":96,"threads_started":1}
W20260806 18:39:52.743948 11692 catalog_manager.cc:4584] TS 1f1ae74e03424b7786debc2b73ba5b45 (127.8.241.193:44857): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b60c3931203e4834ae6917e1e8de2521: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44857: connect: Connection refused (error 111)
I20260806 18:39:52.751053 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 12260
I20260806 18:39:52.769555 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 11664
2026-08-06T18:39:52Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6585 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:52 dist-test-slave-k5rr krb5kdc[12742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:52 dist-test-slave-k5rr krb5kdc[12742](info): set up 2 sockets
Aug 06 18:39:52 dist-test-slave-k5rr krb5kdc[12742](info): commencing operation
krb5kdc: starting...
W20260806 18:39:53.905682 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:39:53 dist-test-slave-k5rr krb5kdc[12742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:39:54.030061 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39979
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:35697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:54.144918 12757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:54.145218 12757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:54.145262 12757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:54.149374 12757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:54.149444 12757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:54.149463 12757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:54.149479 12757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:54.149506 12757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:54.154878 12757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:35697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:54.155972 12757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:54.157083 12757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:54.166030 12766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:54.166117 12757 server_base.cc:1034] running on GCE node
W20260806 18:39:54.166030 12768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:54.166105 12765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:54.166957 12757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:54.168076 12757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:54.169219 12757 hybrid_clock.cc:648] HybridClock initialized: now 1786041594169192 us; error 42 us; skew 500 ppm
Aug 06 18:39:54 dist-test-slave-k5rr krb5kdc[12742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:54.176542 12757 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:39:54.177621 12757 webserver.cc:460] Webserver started at http://127.8.241.254:41269/ using document root <none> and password file <none>
I20260806 18:39:54.178210 12757 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:54.178272 12757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:54.178496 12757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:54.180624 12757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "580188c8cda445a2872741e515fe4b17"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "2c9f88ab64a47e64a32202ef6dafb2fa"
server_key_iv: "8a652c68a610d772389c4c37bf00e042"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.181216 12757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "580188c8cda445a2872741e515fe4b17"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "2c9f88ab64a47e64a32202ef6dafb2fa"
server_key_iv: "8a652c68a610d772389c4c37bf00e042"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.184957 12757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:39:54.187517 12776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.188758 12757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:39:54.188871 12757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "580188c8cda445a2872741e515fe4b17"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "2c9f88ab64a47e64a32202ef6dafb2fa"
server_key_iv: "8a652c68a610d772389c4c37bf00e042"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.188980 12757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:54.199072 12757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:54.201975 12757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:54.202149 12757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:54.208956 12757 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:39979
I20260806 18:39:54.209023 12852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:39979 every 8 connection(s)
I20260806 18:39:54.210002 12757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:54.212991 12853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:54.217257 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 12757
I20260806 18:39:54.217373 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:39:54.217621 9159 external_mini_cluster.cc:1428] Setting key 06b5a2814e8e544e890828c5478598d0
I20260806 18:39:54.219522 12853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17: Bootstrap starting.
I20260806 18:39:54.221933 12853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:54.222697 12853 log.cc:824] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:54.224634 12853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17: No bootstrap required, opened a new log
Aug 06 18:39:54 dist-test-slave-k5rr krb5kdc[12742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:54.229274 12853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580188c8cda445a2872741e515fe4b17" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39979 } }
I20260806 18:39:54.229569 12853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:54.229635 12853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580188c8cda445a2872741e515fe4b17, State: Initialized, Role: FOLLOWER
I20260806 18:39:54.230129 12853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580188c8cda445a2872741e515fe4b17" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39979 } }
I20260806 18:39:54.230278 12853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:54.230346 12853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:54.230458 12853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:54.231415 12853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580188c8cda445a2872741e515fe4b17" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39979 } }
I20260806 18:39:54.231814 12853 leader_election.cc:302] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 580188c8cda445a2872741e515fe4b17; no voters:
I20260806 18:39:54.232142 12853 leader_election.cc:288] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:54.232256 12862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:54.232470 12862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [term 1 LEADER]: Becoming Leader. State: Replica: 580188c8cda445a2872741e515fe4b17, State: Running, Role: LEADER
I20260806 18:39:54.232484 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:54.219083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.8.241.254:39979)
0806 18:39:54.219575 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:54.219592 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:54.219630 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:39:54.220299 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:54.220317 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:54.220681 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:54.221028 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:54.221851 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.222931 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.223764 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.224242 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.224390 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:54.227111 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:54.227119 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:54.227140 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:54.227178 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:54.229215 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:54.229692 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:54.229699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:54.229706 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:54.229857 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:54.230138 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:54.230142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:54.230144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:54.230659 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:54.230875 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:54.231185 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 18:39:54.231518 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":150,"threads_started":1}
I20260806 18:39:54.232921 12862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580188c8cda445a2872741e515fe4b17" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39979 } }
I20260806 18:39:54.233536 12853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:54.234246 12864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 580188c8cda445a2872741e515fe4b17. Latest consensus state: current_term: 1 leader_uuid: "580188c8cda445a2872741e515fe4b17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580188c8cda445a2872741e515fe4b17" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39979 } } }
I20260806 18:39:54.234370 12864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:54.235044 12865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "580188c8cda445a2872741e515fe4b17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580188c8cda445a2872741e515fe4b17" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39979 } } }
I20260806 18:39:54.235134 12865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:54.235421 12874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:54.238692 12874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:54.244541 12874 catalog_manager.cc:1347] Generated new cluster ID: e5ab917d009c491fa262090cf2d09f28
I20260806 18:39:54.244628 12874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:54.256865 12874 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:54.257872 12874 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:54.265689 12874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 580188c8cda445a2872741e515fe4b17: Generated new TSK 0
I20260806 18:39:54.266444 12874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:39:54.295977 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39979
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:35697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:54.433504 12894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:54.433777 12894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:54.433821 12894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:54.437891 12894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:54.437953 12894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:54.438011 12894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:39:54.443331 12894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:35697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:54.444401 12894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:54.445484 12894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:54.455797 12902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:54.455847 12907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:54.455849 12905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:54.456022 12894 server_base.cc:1034] running on GCE node
I20260806 18:39:54.456451 12894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:54.457175 12894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:54.458348 12894 hybrid_clock.cc:648] HybridClock initialized: now 1786041594458317 us; error 45 us; skew 500 ppm
Aug 06 18:39:54 dist-test-slave-k5rr krb5kdc[12742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:54.461234 12894 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:39:54.462252 12894 webserver.cc:460] Webserver started at http://127.8.241.193:33623/ using document root <none> and password file <none>
I20260806 18:39:54.462842 12894 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:54.462911 12894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:54.463155 12894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:54.465394 12894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "33fa7980316141c6ba864e6234937e88"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "3174fec6b5c75801cdaa0ed19dcf18c5"
server_key_iv: "76cba35cb5449d9e3533363c0be6d7be"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.465970 12894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "33fa7980316141c6ba864e6234937e88"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "3174fec6b5c75801cdaa0ed19dcf18c5"
server_key_iv: "76cba35cb5449d9e3533363c0be6d7be"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.469831 12894 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:39:54.472435 12918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.473825 12894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:39:54.473960 12894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "33fa7980316141c6ba864e6234937e88"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "3174fec6b5c75801cdaa0ed19dcf18c5"
server_key_iv: "76cba35cb5449d9e3533363c0be6d7be"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.474071 12894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:54.491122 12894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:54.494623 12894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:54.494817 12894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:54.495455 12894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:54.496359 12894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:54.496407 12894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.496464 12894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:54.496491 12894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.505645 12894 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:33615
I20260806 18:39:54.505681 13090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:33615 every 8 connection(s)
I20260806 18:39:54.506611 12894 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:39:54.514142 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 12894
I20260806 18:39:54.514236 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:39:54.514492 9159 external_mini_cluster.cc:1428] Setting key 1b5ed4ec9fed722be78024fbb7e532ef
Aug 06 18:39:54 dist-test-slave-k5rr krb5kdc[12742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:54.520929 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:54.508486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:42121 (local address 127.8.241.254:39979)
0806 18:39:54.508627 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:54.508631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:54.509766 (+ 1135us) server_negotiation.cc:407] Connection header received
0806 18:39:54.510714 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:54.510718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:54.510777 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:54.510853 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:54.512243 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.512778 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.513987 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.514220 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.514313 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:54.517360 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:54.517363 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:54.517366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:54.517391 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:54.518865 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:54.519384 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:54.519388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:54.519390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:54.519482 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:54.519787 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:54.519790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:54.519791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:54.520018 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:54.520111 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:54.520611 (+ 500us) server_negotiation.cc:299] Negotiation successful
0806 18:39:54.520751 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:39:54.521791 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:54.508884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39979 (local address 127.8.241.193:42121)
0806 18:39:54.509613 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:54.509657 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:54.510498 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:54.511056 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:54.511063 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:54.511425 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:54.512055 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:54.512081 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.512906 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.512908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:54.513843 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:54.513854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.514470 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.514473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:54.514638 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:54.515242 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:54.515263 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:54.517187 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:54.519004 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:54.519009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:54.519024 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:54.519278 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:54.519560 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:54.519561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:54.519563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:54.519688 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:54.520107 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:54.520112 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:54.520353 (+ 241us) client_negotiation.cc:769] Sending connection context
0806 18:39:54.520514 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 18:39:54.520788 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":138,"threads_started":1}
I20260806 18:39:54.523000 13091 heartbeater.cc:347] Connected to a master server at 127.8.241.254:39979
I20260806 18:39:54.523233 13091 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:54.523721 13091 heartbeater.cc:511] Master 127.8.241.254:39979 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:39:54.525393 12803 ts_manager.cc:194] Registered new tserver with Master: 33fa7980316141c6ba864e6234937e88 (127.8.241.193:33615)
I20260806 18:39:54.526544 12803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:42121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:39:54.543435 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39979
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:35697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:54.658861 13103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:54.659138 13103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:54.659183 13103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:54.663218 13103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:54.663292 13103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:54.663354 13103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:39:54.668685 13103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:35697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:54.669829 13103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:54.670881 13103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:54.680794 13111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:54.680809 13114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:54.680833 13108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:54.681021 13103 server_base.cc:1034] running on GCE node
I20260806 18:39:54.681499 13103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:54.682176 13103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:54.683360 13103 hybrid_clock.cc:648] HybridClock initialized: now 1786041594683332 us; error 42 us; skew 500 ppm
Aug 06 18:39:54 dist-test-slave-k5rr krb5kdc[12742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:54.686149 13103 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:39:54.687057 13103 webserver.cc:460] Webserver started at http://127.8.241.194:34579/ using document root <none> and password file <none>
I20260806 18:39:54.687639 13103 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:54.687711 13103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:54.687934 13103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:54.690277 13103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "ee37842a60684b66845d45af6f27c2e2"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "85a779eec5d2f2066495c6eccaa4a55e"
server_key_iv: "6a0cba9f27b4b0e51ad4a7c5c6b6d5d7"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.690892 13103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "ee37842a60684b66845d45af6f27c2e2"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "85a779eec5d2f2066495c6eccaa4a55e"
server_key_iv: "6a0cba9f27b4b0e51ad4a7c5c6b6d5d7"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.695006 13103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:39:54.697670 13123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.699172 13103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:39:54.699299 13103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "ee37842a60684b66845d45af6f27c2e2"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "85a779eec5d2f2066495c6eccaa4a55e"
server_key_iv: "6a0cba9f27b4b0e51ad4a7c5c6b6d5d7"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.699434 13103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:54.716558 13103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:54.719578 13103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:54.719794 13103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:54.720434 13103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:54.721352 13103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:54.721405 13103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.721446 13103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:54.721460 13103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.730073 13103 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:44063
I20260806 18:39:54.730096 13290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:44063 every 8 connection(s)
I20260806 18:39:54.731009 13103 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:39:54.740263 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 13103
I20260806 18:39:54.740391 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
Aug 06 18:39:54 dist-test-slave-k5rr krb5kdc[12742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:54.740746 9159 external_mini_cluster.cc:1428] Setting key af8d53c4eff8d82c4ebfecc6e08e8f74
I20260806 18:39:54.744838 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:54.732886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:55849 (local address 127.8.241.254:39979)
0806 18:39:54.733051 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:54.733056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:54.734247 (+ 1191us) server_negotiation.cc:407] Connection header received
0806 18:39:54.735212 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:54.735217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:54.735304 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:54.735421 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:54.736610 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.737227 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.738328 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.738557 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.738651 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:54.741251 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:54.741255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:54.741257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:54.741285 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:54.742864 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:54.743326 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:54.743329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:54.743331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:54.743420 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:54.743700 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:54.743702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:54.743704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:54.743932 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:54.744029 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:54.744614 (+ 585us) server_negotiation.cc:299] Negotiation successful
0806 18:39:54.744712 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:39:54.745981 13294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:54.733328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39979 (local address 127.8.241.194:55849)
0806 18:39:54.733993 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:54.734156 (+ 163us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:54.734960 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:54.735573 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:54.735580 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:54.735965 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:54.736459 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:54.736474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.737330 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.737332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:54.738206 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:54.738216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.738691 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.738694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:54.738839 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:54.739377 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:54.739394 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:54.741071 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:54.742971 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:54.742976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:54.742988 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:54.743238 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:54.743499 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:54.743501 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:54.743503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:54.743628 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:54.744018 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:54.744025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:54.744350 (+ 325us) client_negotiation.cc:769] Sending connection context
0806 18:39:54.744575 (+ 225us) client_negotiation.cc:240] Negotiation successful
0806 18:39:54.744909 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":111,"threads_started":1}
I20260806 18:39:54.747171 13291 heartbeater.cc:347] Connected to a master server at 127.8.241.254:39979
I20260806 18:39:54.747421 13291 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:54.747921 13291 heartbeater.cc:511] Master 127.8.241.254:39979 requested a full tablet report, sending...
I20260806 18:39:54.749171 12803 ts_manager.cc:194] Registered new tserver with Master: ee37842a60684b66845d45af6f27c2e2 (127.8.241.194:44063)
I20260806 18:39:54.749931 12803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:55849
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:39:54.770223 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39979
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:35697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:54.885540 13297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:54.885815 13297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:54.885876 13297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:54.889875 13297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:54.889946 13297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:54.890025 13297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:39:54.895596 13297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:35697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:54.896799 13297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:54.897961 13297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:54.908102 13303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:54.908174 13297 server_base.cc:1034] running on GCE node
W20260806 18:39:54.908195 13304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:54.908124 13306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:54.908776 13297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:54.909516 13297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:54.910703 13297 hybrid_clock.cc:648] HybridClock initialized: now 1786041594910668 us; error 43 us; skew 500 ppm
Aug 06 18:39:54 dist-test-slave-k5rr krb5kdc[12742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:54.913565 13297 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:39:54.914494 13297 webserver.cc:460] Webserver started at http://127.8.241.195:33195/ using document root <none> and password file <none>
I20260806 18:39:54.915062 13297 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:54.915131 13297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:54.915341 13297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:54.917567 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "eebcbce3be504a80ad82e754bad551e7"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "f6b8be389db1bff3e334581c17e2ebe9"
server_key_iv: "f27b3af81cf543fd25ffe1f9c3eaec02"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.918138 13297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "eebcbce3be504a80ad82e754bad551e7"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "f6b8be389db1bff3e334581c17e2ebe9"
server_key_iv: "f27b3af81cf543fd25ffe1f9c3eaec02"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.922034 13297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:39:54.924567 13315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.925845 13297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:39:54.925978 13297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "eebcbce3be504a80ad82e754bad551e7"
format_stamp: "Formatted at 2026-08-06 18:39:54 on dist-test-slave-k5rr"
server_key: "f6b8be389db1bff3e334581c17e2ebe9"
server_key_iv: "f27b3af81cf543fd25ffe1f9c3eaec02"
server_key_version: "encryptionkey@0"
I20260806 18:39:54.926084 13297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:54.938690 13297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:54.941562 13297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:54.941751 13297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:54.942351 13297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:54.943215 13297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:54.943272 13297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.943327 13297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:54.943356 13297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:54.951997 13297 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:46541
I20260806 18:39:54.952023 13487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:46541 every 8 connection(s)
I20260806 18:39:54.952925 13297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:39:54.956668 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 13297
I20260806 18:39:54.956785 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:39:54.957110 9159 external_mini_cluster.cc:1428] Setting key dc929412b79b95d9c91e72363dc8c1c3
Aug 06 18:39:54 dist-test-slave-k5rr krb5kdc[12742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041594, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:54.967208 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:54.954911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:52631 (local address 127.8.241.254:39979)
0806 18:39:54.955068 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:54.955072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:54.956176 (+ 1104us) server_negotiation.cc:407] Connection header received
0806 18:39:54.957337 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:54.957340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:54.957391 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:54.957493 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:54.958938 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.959539 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.960677 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.960884 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.960960 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:54.963604 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:54.963607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:54.963609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:54.963635 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:54.965106 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:54.965622 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:54.965626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:54.965628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:54.965711 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:54.966054 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:54.966057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:54.966059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:54.966267 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:54.966354 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:54.966909 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 18:39:54.967041 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:39:54.968111 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:54.955352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39979 (local address 127.8.241.195:52631)
0806 18:39:54.956029 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:54.956072 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:54.957046 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:54.957714 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:54.957721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:54.958174 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:54.958798 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:54.958820 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.959668 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.959670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:54.960581 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:54.960591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:54.961030 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:54.961035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:54.961216 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:54.961741 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:54.961758 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:54.963423 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:54.965219 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:54.965226 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:54.965246 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:54.965520 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:54.965798 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:54.965801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:54.965803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:54.965962 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:54.966377 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:54.966384 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:54.966654 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 18:39:54.966849 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 18:39:54.967205 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":117,"threads_started":1}
I20260806 18:39:54.969327 13488 heartbeater.cc:347] Connected to a master server at 127.8.241.254:39979
I20260806 18:39:54.969605 13488 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:54.970201 13488 heartbeater.cc:511] Master 127.8.241.254:39979 requested a full tablet report, sending...
I20260806 18:39:54.971314 12803 ts_manager.cc:194] Registered new tserver with Master: eebcbce3be504a80ad82e754bad551e7 (127.8.241.195:46541)
I20260806 18:39:54.971939 12803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:52631
I20260806 18:39:54.984266 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:39:54.998503 12859 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:39:54.997064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.8.241.254:39979)
0806 18:39:54.997214 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:54.997219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:54.997393 (+ 174us) server_negotiation.cc:407] Connection header received
0806 18:39:54.997522 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:54.997525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:54.997574 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:54.997642 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:54.998363 (+ 721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53468: BlockingRecv error: recv got EOF from 127.0.0.1:53468 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:39:54.999564 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 12894
I20260806 18:39:55.016772 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 13103
I20260806 18:39:55.028628 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 13297
I20260806 18:39:55.039139 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 12757
2026-08-06T18:39:55Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2265 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:55 dist-test-slave-k5rr krb5kdc[13510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:55 dist-test-slave-k5rr krb5kdc[13510](info): set up 2 sockets
Aug 06 18:39:55 dist-test-slave-k5rr krb5kdc[13510](info): commencing operation
krb5kdc: starting...
W20260806 18:39:56.169551 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:39:56 dist-test-slave-k5rr krb5kdc[13510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:39:56.294078 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39417
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:56.408815 13528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:56.409150 13528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:56.409198 13528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:56.413196 13528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:56.413264 13528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:56.413282 13528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:56.413298 13528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:56.413312 13528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:56.418663 13528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:56.419746 13528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:56.420845 13528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:56.429576 13533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:56.429538 13534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:56.429567 13539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:56.429641 13528 server_base.cc:1034] running on GCE node
I20260806 18:39:56.430480 13528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:56.431721 13528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:56.432910 13528 hybrid_clock.cc:648] HybridClock initialized: now 1786041596432882 us; error 41 us; skew 500 ppm
Aug 06 18:39:56 dist-test-slave-k5rr krb5kdc[13510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041596, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:56.436002 13528 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:39:56.437181 13528 webserver.cc:460] Webserver started at http://127.8.241.254:42911/ using document root <none> and password file <none>
I20260806 18:39:56.437799 13528 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:56.437880 13528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:56.438114 13528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:56.440363 13528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7"
format_stamp: "Formatted at 2026-08-06 18:39:56 on dist-test-slave-k5rr"
server_key: "2ba21b256ef55873c0fd5ad138a1cee0"
server_key_iv: "09a4bed1a293acfd11f72f00d59b4656"
server_key_version: "encryptionkey@0"
I20260806 18:39:56.440970 13528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7"
format_stamp: "Formatted at 2026-08-06 18:39:56 on dist-test-slave-k5rr"
server_key: "2ba21b256ef55873c0fd5ad138a1cee0"
server_key_iv: "09a4bed1a293acfd11f72f00d59b4656"
server_key_version: "encryptionkey@0"
I20260806 18:39:56.444941 13528 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:39:56.447579 13548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:56.448936 13528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 18:39:56.449116 13528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7"
format_stamp: "Formatted at 2026-08-06 18:39:56 on dist-test-slave-k5rr"
server_key: "2ba21b256ef55873c0fd5ad138a1cee0"
server_key_iv: "09a4bed1a293acfd11f72f00d59b4656"
server_key_version: "encryptionkey@0"
I20260806 18:39:56.449227 13528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:56.462607 13528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:56.466264 13528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:56.466486 13528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:56.474136 13528 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:39417
I20260806 18:39:56.474395 13644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:39417 every 8 connection(s)
I20260806 18:39:56.475229 13528 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:56.478641 13645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:56.480849 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 13528
I20260806 18:39:56.480983 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:39:56.481288 9159 external_mini_cluster.cc:1428] Setting key 0188310f44df7259ead770fb128be4ca
I20260806 18:39:56.486166 13645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7: Bootstrap starting.
I20260806 18:39:56.489277 13645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:56.490224 13645 log.cc:824] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:56.493391 13645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7: No bootstrap required, opened a new log
Aug 06 18:39:56 dist-test-slave-k5rr krb5kdc[13510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:56.497741 13645 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39417 } }
I20260806 18:39:56.498076 13645 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:56.498137 13645 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f1b6f68c5d4b61b4461f13c2d63ac7, State: Initialized, Role: FOLLOWER
I20260806 18:39:56.498699 13645 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39417 } }
I20260806 18:39:56.498876 13645 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:56.498955 13645 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:56.499075 13645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:56.500388 13645 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39417 } }
I20260806 18:39:56.500833 13645 leader_election.cc:302] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3f1b6f68c5d4b61b4461f13c2d63ac7; no voters:
I20260806 18:39:56.502344 13645 leader_election.cc:288] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:56.502357 13649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:56.482784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.8.241.254:39417)
0806 18:39:56.483353 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:56.483375 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:56.485385 (+ 2010us) server_negotiation.cc:407] Connection header received
0806 18:39:56.486444 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:56.486471 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:56.486928 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:56.487286 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:56.488775 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:56.490318 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:56.491514 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:56.492290 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:56.492476 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:56.495380 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:56.495388 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:56.495411 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:56.495452 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:56.497901 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:56.498519 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:56.498527 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:56.498534 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:56.498718 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:56.499226 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:56.499230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:56.499232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:56.499791 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:56.500809 (+ 1018us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:56.500999 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 18:39:56.501370 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":150,"threads_started":1}
I20260806 18:39:56.502729 13651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:56.502967 13651 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [term 1 LEADER]: Becoming Leader. State: Replica: e3f1b6f68c5d4b61b4461f13c2d63ac7, State: Running, Role: LEADER
I20260806 18:39:56.503340 13651 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39417 } }
I20260806 18:39:56.503803 13645 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:56.504747 13651 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3f1b6f68c5d4b61b4461f13c2d63ac7. Latest consensus state: current_term: 1 leader_uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39417 } } }
I20260806 18:39:56.504877 13651 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:56.505648 13655 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3f1b6f68c5d4b61b4461f13c2d63ac7" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39417 } } }
I20260806 18:39:56.505772 13655 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:56.506016 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 18:39:56.509544 13677 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:56.509654 13677 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:56.509738 13669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:56.510725 13669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:56.519049 13669 catalog_manager.cc:1347] Generated new cluster ID: 2d85804ab104482192e704823ecd8943
I20260806 18:39:56.519147 13669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:56.533381 13649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:56.527259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.8.241.254:39417)
0806 18:39:56.527379 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:56.527384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:56.527720 (+ 336us) server_negotiation.cc:407] Connection header received
0806 18:39:56.527929 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:56.527933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:56.528036 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:56.528115 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:56.528789 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:56.529630 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:56.531076 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:56.531329 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:56.531425 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:56.532328 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:56.532332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:56.532335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:39:56.532870 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:56.532904 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:39:56.532922 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:56.533039 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:56.533146 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 18:39:56.533224 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:39:56.545122 13669 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:56.546490 13669 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:56.555145 13669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3f1b6f68c5d4b61b4461f13c2d63ac7: Generated new TSK 0
I20260806 18:39:56.556131 13669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:39:56.570506 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 13528
2026-08-06T18:39:56Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1534 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:56 dist-test-slave-k5rr krb5kdc[13699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:56 dist-test-slave-k5rr krb5kdc[13699](info): set up 2 sockets
Aug 06 18:39:56 dist-test-slave-k5rr krb5kdc[13699](info): commencing operation
krb5kdc: starting...
W20260806 18:39:57.701349 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:39:57 dist-test-slave-k5rr krb5kdc[13699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:39:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:39:57.827737 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:43991
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:43991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:57.945509 13719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:57.945822 13719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:57.945870 13719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:57.949895 13719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:39:57.949977 13719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:57.949998 13719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:57.950017 13719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:39:57.950034 13719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:39:57.955488 13719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:43991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:43991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:57.956593 13719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:57.957652 13719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:57.966552 13726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:57.966553 13725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:57.966553 13730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:57.966809 13719 server_base.cc:1034] running on GCE node
I20260806 18:39:57.967473 13719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:57.968695 13719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:57.969856 13719 hybrid_clock.cc:648] HybridClock initialized: now 1786041597969840 us; error 32 us; skew 500 ppm
Aug 06 18:39:57 dist-test-slave-k5rr krb5kdc[13699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041597, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:57.972647 13719 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:39:57.973702 13719 webserver.cc:460] Webserver started at http://127.8.241.254:46325/ using document root <none> and password file <none>
I20260806 18:39:57.974330 13719 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:57.974404 13719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:57.974644 13719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:57.976866 13719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "84ba3e92ab55429fa1338d986e5d56a3"
format_stamp: "Formatted at 2026-08-06 18:39:57 on dist-test-slave-k5rr"
server_key: "9fe30c5aa4ace70112fd387d78ae374d"
server_key_iv: "6e27de29e6014778b6d0c2ad4882d355"
server_key_version: "encryptionkey@0"
I20260806 18:39:57.977499 13719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "84ba3e92ab55429fa1338d986e5d56a3"
format_stamp: "Formatted at 2026-08-06 18:39:57 on dist-test-slave-k5rr"
server_key: "9fe30c5aa4ace70112fd387d78ae374d"
server_key_iv: "6e27de29e6014778b6d0c2ad4882d355"
server_key_version: "encryptionkey@0"
I20260806 18:39:57.981505 13719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:39:57.984054 13741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:57.985394 13719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:39:57.985531 13719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "84ba3e92ab55429fa1338d986e5d56a3"
format_stamp: "Formatted at 2026-08-06 18:39:57 on dist-test-slave-k5rr"
server_key: "9fe30c5aa4ace70112fd387d78ae374d"
server_key_iv: "6e27de29e6014778b6d0c2ad4882d355"
server_key_version: "encryptionkey@0"
I20260806 18:39:57.985641 13719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:58.006290 13719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:58.009285 13719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:58.009490 13719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:58.016803 13719 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:43991
I20260806 18:39:58.017118 13818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:43991 every 8 connection(s)
I20260806 18:39:58.017887 13719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:39:58.020905 13820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:39:58.025043 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 13719
I20260806 18:39:58.025159 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:39:58.025413 9159 external_mini_cluster.cc:1428] Setting key b5c926708e86cd2b38d7125752841d67
I20260806 18:39:58.027285 13820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3: Bootstrap starting.
I20260806 18:39:58.029992 13820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3: Neither blocks nor log segments found. Creating new log.
I20260806 18:39:58.030771 13820 log.cc:824] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:39:58.033116 13820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3: No bootstrap required, opened a new log
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[13699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:58.037529 13820 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ba3e92ab55429fa1338d986e5d56a3" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43991 } }
I20260806 18:39:58.037859 13820 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:39:58.037931 13820 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84ba3e92ab55429fa1338d986e5d56a3, State: Initialized, Role: FOLLOWER
I20260806 18:39:58.038403 13820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ba3e92ab55429fa1338d986e5d56a3" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43991 } }
I20260806 18:39:58.038582 13820 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:39:58.038659 13820 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:39:58.038775 13820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:39:58.039804 13820 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ba3e92ab55429fa1338d986e5d56a3" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43991 } }
I20260806 18:39:58.040084 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:58.026882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.8.241.254:43991)
0806 18:39:58.027313 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:58.027330 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:58.027373 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:39:58.028053 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:58.028073 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:58.028419 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:58.028720 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:58.029674 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.030740 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.031763 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.032245 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.032370 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:58.035199 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:58.035205 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:58.035222 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:58.035252 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:58.036893 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.037421 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.037427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.037432 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.037564 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.037858 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.037861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.037863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.038382 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:58.038594 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:58.038917 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 18:39:58.039247 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":106,"threads_started":1}
I20260806 18:39:58.040164 13820 leader_election.cc:302] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84ba3e92ab55429fa1338d986e5d56a3; no voters:
I20260806 18:39:58.040515 13820 leader_election.cc:288] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:39:58.040620 13825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:39:58.040838 13825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [term 1 LEADER]: Becoming Leader. State: Replica: 84ba3e92ab55429fa1338d986e5d56a3, State: Running, Role: LEADER
I20260806 18:39:58.041174 13825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ba3e92ab55429fa1338d986e5d56a3" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43991 } }
I20260806 18:39:58.041833 13820 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:39:58.042387 13827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84ba3e92ab55429fa1338d986e5d56a3. Latest consensus state: current_term: 1 leader_uuid: "84ba3e92ab55429fa1338d986e5d56a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ba3e92ab55429fa1338d986e5d56a3" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43991 } } }
I20260806 18:39:58.042523 13827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:39:58.042837 13826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84ba3e92ab55429fa1338d986e5d56a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84ba3e92ab55429fa1338d986e5d56a3" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43991 } } }
I20260806 18:39:58.042923 13826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3 [sys.catalog]: This master's current role is: LEADER
W20260806 18:39:58.046720 13847 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:39:58.046792 13847 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:39:58.046866 13830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:39:58.047720 13830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:39:58.053859 13830 catalog_manager.cc:1347] Generated new cluster ID: bc5a8445310b40479a4b9377407db7ab
I20260806 18:39:58.053918 13830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:39:58.064914 13830 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:39:58.065932 13830 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:39:58.075501 13830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84ba3e92ab55429fa1338d986e5d56a3: Generated new TSK 0
I20260806 18:39:58.076300 13830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:39:58.106138 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43991
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:58.255163 13854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:58.255465 13854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:58.255524 13854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:58.259665 13854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:58.259747 13854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:58.259830 13854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:39:58.265455 13854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:58.266644 13854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:58.267781 13854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:58.277818 13859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:58.277966 13854 server_base.cc:1034] running on GCE node
W20260806 18:39:58.277787 13860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:58.278046 13867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:58.278564 13854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:58.279202 13854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:58.280360 13854 hybrid_clock.cc:648] HybridClock initialized: now 1786041598280336 us; error 35 us; skew 500 ppm
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[13699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041598, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:58.283083 13854 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:39:58.283986 13854 webserver.cc:460] Webserver started at http://127.8.241.193:43171/ using document root <none> and password file <none>
I20260806 18:39:58.284535 13854 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:58.284595 13854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:58.284770 13854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:58.286886 13854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "8969a48097db40f48b9436541f8dd464"
format_stamp: "Formatted at 2026-08-06 18:39:58 on dist-test-slave-k5rr"
server_key: "18363d0df79b922579336d9e8cbc26b7"
server_key_iv: "59d37e6b01e0f4bf83daa7a8af5f37f1"
server_key_version: "encryptionkey@0"
I20260806 18:39:58.287417 13854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "8969a48097db40f48b9436541f8dd464"
format_stamp: "Formatted at 2026-08-06 18:39:58 on dist-test-slave-k5rr"
server_key: "18363d0df79b922579336d9e8cbc26b7"
server_key_iv: "59d37e6b01e0f4bf83daa7a8af5f37f1"
server_key_version: "encryptionkey@0"
I20260806 18:39:58.291210 13854 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:39:58.293733 13875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:58.295037 13854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:39:58.295179 13854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "8969a48097db40f48b9436541f8dd464"
format_stamp: "Formatted at 2026-08-06 18:39:58 on dist-test-slave-k5rr"
server_key: "18363d0df79b922579336d9e8cbc26b7"
server_key_iv: "59d37e6b01e0f4bf83daa7a8af5f37f1"
server_key_version: "encryptionkey@0"
I20260806 18:39:58.295317 13854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:58.320343 13854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:58.323374 13854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:58.323580 13854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:58.324275 13854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:58.325246 13854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:58.325297 13854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:58.325343 13854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:58.325371 13854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:58.333974 13854 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:38803
I20260806 18:39:58.334007 14039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:38803 every 8 connection(s)
I20260806 18:39:58.334929 13854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:39:58.344807 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 13854
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[13699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041598, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:58.344918 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:39:58.345279 9159 external_mini_cluster.cc:1428] Setting key 321c1727ddb1b80f531947b4a6960c9d
I20260806 18:39:58.349426 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:58.336788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:33177 (local address 127.8.241.254:43991)
0806 18:39:58.336943 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:58.336948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:58.337980 (+ 1032us) server_negotiation.cc:407] Connection header received
0806 18:39:58.339337 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:58.339340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:58.339391 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:58.339468 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:58.340817 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.341413 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.342650 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.342892 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.342979 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:58.345617 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:58.345620 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:58.345622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:58.345647 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:58.347010 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.347647 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.347650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.347651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.347730 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.348126 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.348128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.348129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.348303 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:58.348374 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:58.349116 (+ 742us) server_negotiation.cc:299] Negotiation successful
0806 18:39:58.349257 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:39:58.350286 14042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:58.337265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43991 (local address 127.8.241.193:33177)
0806 18:39:58.337784 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:58.337855 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:58.339119 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:58.339704 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:58.339714 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:58.340104 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:58.340650 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:58.340665 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.341533 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.341536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:58.342551 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:58.342564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.343032 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.343035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:58.343273 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:58.343819 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:58.343836 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:58.345435 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:58.347128 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:58.347136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:58.347156 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:58.347541 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:58.347809 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:58.347812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:58.347814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:58.348043 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:58.348399 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:58.348407 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:58.348771 (+ 364us) client_negotiation.cc:769] Sending connection context
0806 18:39:58.349065 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 18:39:58.349377 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":140,"threads_started":1}
I20260806 18:39:58.351583 14040 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43991
I20260806 18:39:58.351842 14040 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:58.352546 14040 heartbeater.cc:511] Master 127.8.241.254:43991 requested a full tablet report, sending...
I20260806 18:39:58.354205 13767 ts_manager.cc:194] Registered new tserver with Master: 8969a48097db40f48b9436541f8dd464 (127.8.241.193:38803)
I20260806 18:39:58.355345 13767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:33177
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:39:58.373646 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43991
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:58.489509 14046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:58.489830 14046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:58.489893 14046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:58.494280 14046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:58.494359 14046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:58.494446 14046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:39:58.500101 14046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:58.501325 14046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:58.502393 14046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:58.512508 14056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:58.512513 14057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:58.512697 14046 server_base.cc:1034] running on GCE node
W20260806 18:39:58.512512 14059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:58.513265 14046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:58.513927 14046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:58.515101 14046 hybrid_clock.cc:648] HybridClock initialized: now 1786041598515078 us; error 35 us; skew 500 ppm
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[13699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041598, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:58.518508 14046 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:39:58.519573 14046 webserver.cc:460] Webserver started at http://127.8.241.194:41745/ using document root <none> and password file <none>
I20260806 18:39:58.520211 14046 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:58.520278 14046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:58.520490 14046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:58.522665 14046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "8e0ae6c509ef445798d3b32e10dcf669"
format_stamp: "Formatted at 2026-08-06 18:39:58 on dist-test-slave-k5rr"
server_key: "e874dcdcf33ddb19116a3889bdea5eab"
server_key_iv: "9eaea2178a6be0d1d0eaee5a307c9d89"
server_key_version: "encryptionkey@0"
I20260806 18:39:58.523223 14046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "8e0ae6c509ef445798d3b32e10dcf669"
format_stamp: "Formatted at 2026-08-06 18:39:58 on dist-test-slave-k5rr"
server_key: "e874dcdcf33ddb19116a3889bdea5eab"
server_key_iv: "9eaea2178a6be0d1d0eaee5a307c9d89"
server_key_version: "encryptionkey@0"
I20260806 18:39:58.527215 14046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:39:58.529759 14068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:58.531116 14046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:39:58.531255 14046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "8e0ae6c509ef445798d3b32e10dcf669"
format_stamp: "Formatted at 2026-08-06 18:39:58 on dist-test-slave-k5rr"
server_key: "e874dcdcf33ddb19116a3889bdea5eab"
server_key_iv: "9eaea2178a6be0d1d0eaee5a307c9d89"
server_key_version: "encryptionkey@0"
I20260806 18:39:58.531366 14046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:58.543510 14046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:58.547086 14046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:58.547276 14046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:58.547890 14046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:58.548794 14046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:58.548843 14046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:58.548898 14046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:58.548923 14046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:58.557601 14046 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:37117
I20260806 18:39:58.557626 14233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:37117 every 8 connection(s)
I20260806 18:39:58.558570 14046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:39:58.560796 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 14046
I20260806 18:39:58.560899 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:39:58.561218 9159 external_mini_cluster.cc:1428] Setting key c25ef6f6d917f1333b4012a397c07481
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[13699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041598, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:39:58.573999 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:58.560446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:56963 (local address 127.8.241.254:43991)
0806 18:39:58.560654 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:58.560659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:58.561661 (+ 1002us) server_negotiation.cc:407] Connection header received
0806 18:39:58.563388 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:58.563391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:58.563442 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:58.563527 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:58.565133 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.565667 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.567137 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.567375 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.567472 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:58.570392 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:58.570395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:58.570397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:58.570424 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:58.571804 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.572427 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.572430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.572431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.572530 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.572844 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.572846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.572847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.573077 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:58.573196 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:58.573742 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 18:39:58.573863 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
I20260806 18:39:58.574976 14238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:58.560857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43991 (local address 127.8.241.194:56963)
0806 18:39:58.561469 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:58.561528 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:58.563114 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:58.563733 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:58.563743 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:58.564260 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:58.564968 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:58.564986 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.565795 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.565799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:58.567012 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:58.567024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.567543 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.567548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:58.567778 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:58.568388 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:58.568409 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:58.570188 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:58.571927 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:58.571935 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:58.571953 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:58.572328 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:58.572616 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:58.572618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:58.572620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:58.572771 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:58.573203 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:58.573211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:58.573506 (+ 295us) client_negotiation.cc:769] Sending connection context
0806 18:39:58.573678 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 18:39:58.573973 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":164,"threads_started":1}
I20260806 18:39:58.576257 14234 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43991
I20260806 18:39:58.576526 14234 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:58.577144 14234 heartbeater.cc:511] Master 127.8.241.254:43991 requested a full tablet report, sending...
I20260806 18:39:58.578347 13767 ts_manager.cc:194] Registered new tserver with Master: 8e0ae6c509ef445798d3b32e10dcf669 (127.8.241.194:37117)
I20260806 18:39:58.578964 13767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:56963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:39:58.590349 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43991
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:39:58.706919 14243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:39:58.707214 14243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:39:58.707261 14243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:39:58.711349 14243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:39:58.711426 14243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:39:58.711503 14243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:39:58.717162 14243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:39:58.718320 14243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:39:58.719393 14243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:39:58.729526 14253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:58.729530 14251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:39:58.729559 14248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:39:58.730021 14243 server_base.cc:1034] running on GCE node
I20260806 18:39:58.730428 14243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:39:58.731083 14243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:39:58.732240 14243 hybrid_clock.cc:648] HybridClock initialized: now 1786041598732221 us; error 28 us; skew 500 ppm
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[13699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041598, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:39:58.735062 14243 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:39:58.736063 14243 webserver.cc:460] Webserver started at http://127.8.241.195:39427/ using document root <none> and password file <none>
I20260806 18:39:58.736653 14243 fs_manager.cc:359] Metadata directory not provided
I20260806 18:39:58.736716 14243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:39:58.736892 14243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:39:58.739084 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "a0d11a22465f43e99891a6d8160df051"
format_stamp: "Formatted at 2026-08-06 18:39:58 on dist-test-slave-k5rr"
server_key: "7208cdc53c010e654570abf19ca140a2"
server_key_iv: "6c0b7e4d5ab5269daf11021b3429fabb"
server_key_version: "encryptionkey@0"
I20260806 18:39:58.739627 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "a0d11a22465f43e99891a6d8160df051"
format_stamp: "Formatted at 2026-08-06 18:39:58 on dist-test-slave-k5rr"
server_key: "7208cdc53c010e654570abf19ca140a2"
server_key_iv: "6c0b7e4d5ab5269daf11021b3429fabb"
server_key_version: "encryptionkey@0"
I20260806 18:39:58.743703 14243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:39:58.746444 14260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:58.747749 14243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:39:58.747870 14243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "a0d11a22465f43e99891a6d8160df051"
format_stamp: "Formatted at 2026-08-06 18:39:58 on dist-test-slave-k5rr"
server_key: "7208cdc53c010e654570abf19ca140a2"
server_key_iv: "6c0b7e4d5ab5269daf11021b3429fabb"
server_key_version: "encryptionkey@0"
I20260806 18:39:58.747982 14243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:39:58.763386 14243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:39:58.766680 14243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:39:58.766868 14243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:39:58.767493 14243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:39:58.768396 14243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:39:58.768467 14243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:58.768533 14243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:39:58.768559 14243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:39:58.777490 14243 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:34513
I20260806 18:39:58.777509 14431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:34513 every 8 connection(s)
I20260806 18:39:58.778488 14243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[13699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041598, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:58.788244 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 14243
I20260806 18:39:58.788383 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:39:58.788664 9159 external_mini_cluster.cc:1428] Setting key 5822e7ef162b244f6f5a81dbb68b6a88
I20260806 18:39:58.792227 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:58.780175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:49511 (local address 127.8.241.254:43991)
0806 18:39:58.780317 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:58.780320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:58.781206 (+ 886us) server_negotiation.cc:407] Connection header received
0806 18:39:58.782616 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:58.782619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:58.782672 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:58.782747 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:58.783963 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.784486 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.785598 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.785812 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.785888 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:58.788421 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:58.788425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:58.788428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:58.788459 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:58.790123 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.790686 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.790690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.790692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.790779 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.791191 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.791193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.791195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.791386 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:58.791481 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:58.792001 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 18:39:58.792102 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:39:58.793148 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:58.780545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43991 (local address 127.8.241.195:49511)
0806 18:39:58.780997 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0806 18:39:58.781085 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0806 18:39:58.782409 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:39:58.782906 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:39:58.782914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:39:58.783312 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:39:58.783835 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:58.783850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.784579 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.784582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:58.785504 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:39:58.785514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.785930 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.785933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:39:58.786071 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:58.786680 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:39:58.786697 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:39:58.788207 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:39:58.790250 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:58.790256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:58.790275 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:58.790595 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:58.790890 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:39:58.790893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:39:58.790895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:39:58.791103 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:39:58.791518 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:39:58.791524 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:39:58.791785 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 18:39:58.791959 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 18:39:58.792238 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":122,"threads_started":1}
I20260806 18:39:58.794255 14433 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43991
I20260806 18:39:58.794503 14433 heartbeater.cc:464] Registering TS with master...
I20260806 18:39:58.795008 14433 heartbeater.cc:511] Master 127.8.241.254:43991 requested a full tablet report, sending...
I20260806 18:39:58.796028 13767 ts_manager.cc:194] Registered new tserver with Master: a0d11a22465f43e99891a6d8160df051 (127.8.241.195:34513)
I20260806 18:39:58.796604 13767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:49511
I20260806 18:39:58.802678 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[13699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041598, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[13699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041598, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:39:58.828141 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:39:58.820272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38560 (local address 127.8.241.254:43991)
0806 18:39:58.820404 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:39:58.820407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:39:58.820577 (+ 170us) server_negotiation.cc:407] Connection header received
0806 18:39:58.820722 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:39:58.820725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:39:58.820771 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:39:58.820833 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:39:58.821552 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.822066 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.822824 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:39:58.823028 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:39:58.823097 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:39:58.825416 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:39:58.825420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:39:58.825422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:39:58.825461 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:39:58.826608 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.827012 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.827016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.827017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.827109 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:39:58.827387 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:39:58.827390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:39:58.827391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:39:58.827630 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:39:58.827778 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:39:58.827909 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 18:39:58.828012 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 18:39:58.828570 13767 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38560
I20260806 18:39:58.829567 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 13854
I20260806 18:39:58.849248 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 14046
I20260806 18:39:58.861344 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 14243
I20260806 18:39:58.873395 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 13719
2026-08-06T18:39:58Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2304 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[14452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[14452](info): set up 2 sockets
Aug 06 18:39:58 dist-test-slave-k5rr krb5kdc[14452](info): commencing operation
krb5kdc: starting...
W20260806 18:40:00.013502 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:40:00 dist-test-slave-k5rr krb5kdc[14452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:40:00.141191 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:35879
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:35879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:00.258025 14469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:00.258330 14469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:00.258389 14469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:00.262502 14469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:00.262583 14469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:00.262611 14469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:00.262632 14469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:00.262655 14469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:00.268213 14469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:35879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:35879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:00.269456 14469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:00.270577 14469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:00.280218 14477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:00.280210 14480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:00.280328 14469 server_base.cc:1034] running on GCE node
W20260806 18:40:00.280218 14476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:00.281201 14469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:00.282469 14469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:00.283667 14469 hybrid_clock.cc:648] HybridClock initialized: now 1786041600283647 us; error 33 us; skew 500 ppm
Aug 06 18:40:00 dist-test-slave-k5rr krb5kdc[14452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:00.314018 14469 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:40:00.315165 14469 webserver.cc:460] Webserver started at http://127.8.241.254:43529/ using document root <none> and password file <none>
I20260806 18:40:00.315798 14469 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:00.315874 14469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:00.316102 14469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:00.318370 14469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "625161689c0548beb5892f09289548d1"
format_stamp: "Formatted at 2026-08-06 18:40:00 on dist-test-slave-k5rr"
server_key: "c19cbc9e49b4e8fb89898f2d4cce1677"
server_key_iv: "12daf26e7429b9f96991ed4de9bb641e"
server_key_version: "encryptionkey@0"
I20260806 18:40:00.318981 14469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "625161689c0548beb5892f09289548d1"
format_stamp: "Formatted at 2026-08-06 18:40:00 on dist-test-slave-k5rr"
server_key: "c19cbc9e49b4e8fb89898f2d4cce1677"
server_key_iv: "12daf26e7429b9f96991ed4de9bb641e"
server_key_version: "encryptionkey@0"
I20260806 18:40:00.323047 14469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:40:00.325626 14493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:00.326953 14469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:00.327095 14469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "625161689c0548beb5892f09289548d1"
format_stamp: "Formatted at 2026-08-06 18:40:00 on dist-test-slave-k5rr"
server_key: "c19cbc9e49b4e8fb89898f2d4cce1677"
server_key_iv: "12daf26e7429b9f96991ed4de9bb641e"
server_key_version: "encryptionkey@0"
I20260806 18:40:00.327204 14469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:00.341914 14469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:00.344928 14469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:00.345165 14469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:00.352476 14469 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:35879
I20260806 18:40:00.352496 14574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:35879 every 8 connection(s)
I20260806 18:40:00.353587 14469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:00.356524 14575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:00.359368 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 14469
I20260806 18:40:00.359498 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:00.359812 9159 external_mini_cluster.cc:1428] Setting key ebb696b4639ec2d1a3a3a50766e43c5d
I20260806 18:40:00.363029 14575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1: Bootstrap starting.
I20260806 18:40:00.365916 14575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:00.366823 14575 log.cc:824] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:40:00 dist-test-slave-k5rr krb5kdc[14452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:00.369472 14575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1: No bootstrap required, opened a new log
I20260806 18:40:00.374249 14575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625161689c0548beb5892f09289548d1" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35879 } }
I20260806 18:40:00.374600 14575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:00.374677 14575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625161689c0548beb5892f09289548d1, State: Initialized, Role: FOLLOWER
I20260806 18:40:00.375214 14575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625161689c0548beb5892f09289548d1" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35879 } }
I20260806 18:40:00.375401 14575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:00.375479 14575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:00.375583 14575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:00.375968 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:00.361385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51030 (local address 127.8.241.254:35879)
0806 18:40:00.361824 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:00.361843 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:00.361883 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:40:00.362550 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:00.362568 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:00.362933 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:00.363281 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:00.364155 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.365261 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.366453 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.367047 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.367194 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:00.370071 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:00.370078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:00.370097 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:00.370130 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:00.372130 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:00.372635 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:00.372642 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:00.372649 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:00.372805 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:00.373317 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:00.373321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:00.373323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:00.373930 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:00.374154 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:00.374472 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 18:40:00.374816 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":125,"threads_started":1}
I20260806 18:40:00.376714 14575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625161689c0548beb5892f09289548d1" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35879 } }
I20260806 18:40:00.377116 14575 leader_election.cc:302] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 625161689c0548beb5892f09289548d1; no voters:
I20260806 18:40:00.377485 14575 leader_election.cc:288] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:00.377590 14583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:00.377853 14583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [term 1 LEADER]: Becoming Leader. State: Replica: 625161689c0548beb5892f09289548d1, State: Running, Role: LEADER
I20260806 18:40:00.378175 14583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625161689c0548beb5892f09289548d1" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35879 } }
I20260806 18:40:00.378826 14575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:00.379416 14590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 625161689c0548beb5892f09289548d1. Latest consensus state: current_term: 1 leader_uuid: "625161689c0548beb5892f09289548d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625161689c0548beb5892f09289548d1" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35879 } } }
I20260806 18:40:00.379549 14590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:00.379887 14588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "625161689c0548beb5892f09289548d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "625161689c0548beb5892f09289548d1" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35879 } } }
I20260806 18:40:00.379981 14588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:00.380053 14596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:00.383329 14596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:00.389815 14596 catalog_manager.cc:1347] Generated new cluster ID: 4ca5375c488f45bd93aafae1c06ca75d
I20260806 18:40:00.389902 14596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:00.413424 14596 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:00.414449 14596 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:00.428203 14596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 625161689c0548beb5892f09289548d1: Generated new TSK 0
I20260806 18:40:00.429054 14596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:40:00.467679 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35879
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:00.588327 14621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:00.588614 14621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:00.588660 14621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:00.593210 14621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:00.593284 14621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:00.593361 14621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:00.598888 14621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:00.600030 14621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:00.601163 14621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:00.611399 14633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:00.611409 14630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:00.611543 14621 server_base.cc:1034] running on GCE node
W20260806 18:40:00.611402 14628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:00.612206 14621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:00.612905 14621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:00.614063 14621 hybrid_clock.cc:648] HybridClock initialized: now 1786041600614040 us; error 35 us; skew 500 ppm
Aug 06 18:40:00 dist-test-slave-k5rr krb5kdc[14452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:00.616781 14621 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:40:00.617723 14621 webserver.cc:460] Webserver started at http://127.8.241.193:45593/ using document root <none> and password file <none>
I20260806 18:40:00.618294 14621 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:00.618361 14621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:00.618588 14621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:00.620685 14621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "5b4bbac2cdf14ae99c5aaf63ff3d54d2"
format_stamp: "Formatted at 2026-08-06 18:40:00 on dist-test-slave-k5rr"
server_key: "d99899a7e20e32a40a09574ce6285cb1"
server_key_iv: "05dfad056fedcff641f892c5551a1d18"
server_key_version: "encryptionkey@0"
I20260806 18:40:00.621269 14621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "5b4bbac2cdf14ae99c5aaf63ff3d54d2"
format_stamp: "Formatted at 2026-08-06 18:40:00 on dist-test-slave-k5rr"
server_key: "d99899a7e20e32a40a09574ce6285cb1"
server_key_iv: "05dfad056fedcff641f892c5551a1d18"
server_key_version: "encryptionkey@0"
I20260806 18:40:00.625218 14621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:40:00.627789 14644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:00.629225 14621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:40:00.629355 14621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "5b4bbac2cdf14ae99c5aaf63ff3d54d2"
format_stamp: "Formatted at 2026-08-06 18:40:00 on dist-test-slave-k5rr"
server_key: "d99899a7e20e32a40a09574ce6285cb1"
server_key_iv: "05dfad056fedcff641f892c5551a1d18"
server_key_version: "encryptionkey@0"
I20260806 18:40:00.629472 14621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:00.651710 14621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:00.654944 14621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:00.655140 14621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:00.655764 14621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:00.656646 14621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:00.656695 14621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:00.656750 14621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:00.656777 14621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:00.665782 14621 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:40589
I20260806 18:40:00.665813 14816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:40589 every 8 connection(s)
I20260806 18:40:00.666738 14621 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:00.675297 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 14621
I20260806 18:40:00.675451 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:00.675786 9159 external_mini_cluster.cc:1428] Setting key f3b2b38dc824188e20237d66cc02769b
Aug 06 18:40:00 dist-test-slave-k5rr krb5kdc[14452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:00.681205 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:00.668758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:42661 (local address 127.8.241.254:35879)
0806 18:40:00.668906 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:00.668941 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:00.669940 (+ 999us) server_negotiation.cc:407] Connection header received
0806 18:40:00.671317 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:00.671319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:00.671370 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:00.671445 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:00.672726 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.673345 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.674523 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.674741 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.674816 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:00.677676 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:00.677679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:00.677682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:00.677705 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:00.678995 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:00.679510 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:00.679513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:00.679514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:00.679588 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:00.679950 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:00.679953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:00.679954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:00.680163 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:00.680261 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:00.680875 (+ 614us) server_negotiation.cc:299] Negotiation successful
0806 18:40:00.681040 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:40:00.682119 14821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:00.669228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:35879 (local address 127.8.241.193:42661)
0806 18:40:00.669766 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:00.669815 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:00.671111 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:00.671630 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:00.671637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:00.672046 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:00.672566 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:00.672581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.673486 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.673490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:00.674408 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:00.674418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.674866 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.674869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:00.675088 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:00.675713 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:00.675735 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:00.677508 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:00.679106 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:00.679113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:00.679132 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:00.679389 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:00.679706 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:00.679709 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:00.679712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:00.679852 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:00.680265 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:00.680273 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:00.680612 (+ 339us) client_negotiation.cc:769] Sending connection context
0806 18:40:00.680814 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 18:40:00.681216 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":127,"threads_started":1}
I20260806 18:40:00.683422 14818 heartbeater.cc:347] Connected to a master server at 127.8.241.254:35879
I20260806 18:40:00.683743 14818 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:00.684454 14818 heartbeater.cc:511] Master 127.8.241.254:35879 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:40:00.686517 14518 ts_manager.cc:194] Registered new tserver with Master: 5b4bbac2cdf14ae99c5aaf63ff3d54d2 (127.8.241.193:40589)
I20260806 18:40:00.687750 14518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:42661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:40:00.704136 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35879
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:00.822137 14825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:00.822422 14825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:00.822471 14825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:00.826597 14825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:00.826675 14825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:00.826746 14825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:00.832393 14825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:00.833611 14825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:00.834695 14825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:40:00.844707 14825 server_base.cc:1034] running on GCE node
W20260806 18:40:00.844655 14837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:00.844659 14835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:00.844668 14833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:00.845307 14825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:00.846000 14825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:00.847179 14825 hybrid_clock.cc:648] HybridClock initialized: now 1786041600847158 us; error 33 us; skew 500 ppm
Aug 06 18:40:00 dist-test-slave-k5rr krb5kdc[14452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:00.850194 14825 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:40:00.851199 14825 webserver.cc:460] Webserver started at http://127.8.241.194:38933/ using document root <none> and password file <none>
I20260806 18:40:00.851816 14825 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:00.851888 14825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:00.852103 14825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:00.854357 14825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "64e243e7a26e48c0a5ee76255f8bea79"
format_stamp: "Formatted at 2026-08-06 18:40:00 on dist-test-slave-k5rr"
server_key: "68813a785e0e80ce1d100de4fd0dc500"
server_key_iv: "f431edb51f6be52328bec5875ec6be66"
server_key_version: "encryptionkey@0"
I20260806 18:40:00.854959 14825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "64e243e7a26e48c0a5ee76255f8bea79"
format_stamp: "Formatted at 2026-08-06 18:40:00 on dist-test-slave-k5rr"
server_key: "68813a785e0e80ce1d100de4fd0dc500"
server_key_iv: "f431edb51f6be52328bec5875ec6be66"
server_key_version: "encryptionkey@0"
I20260806 18:40:00.859277 14825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:40:00.861836 14845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:00.863125 14825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:00.863263 14825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "64e243e7a26e48c0a5ee76255f8bea79"
format_stamp: "Formatted at 2026-08-06 18:40:00 on dist-test-slave-k5rr"
server_key: "68813a785e0e80ce1d100de4fd0dc500"
server_key_iv: "f431edb51f6be52328bec5875ec6be66"
server_key_version: "encryptionkey@0"
I20260806 18:40:00.863373 14825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:00.881793 14825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:00.884940 14825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:00.885176 14825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:00.885812 14825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:00.886759 14825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:00.886814 14825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:00.886870 14825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:00.886897 14825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:00.895761 14825 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:46283
I20260806 18:40:00.895781 15010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:46283 every 8 connection(s)
I20260806 18:40:00.896761 14825 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:00.901350 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 14825
I20260806 18:40:00.901484 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:40:00.901779 9159 external_mini_cluster.cc:1428] Setting key 42ab10527424aae4373a27ced727ef2a
Aug 06 18:40:00 dist-test-slave-k5rr krb5kdc[14452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041600, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:00.911959 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:00.898706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:35459 (local address 127.8.241.254:35879)
0806 18:40:00.898832 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:00.898836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:00.899779 (+ 943us) server_negotiation.cc:407] Connection header received
0806 18:40:00.901278 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:00.901281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:00.901338 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:00.901413 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:00.903030 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.903654 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.905318 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.905652 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.905777 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:00.908232 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:00.908237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:00.908240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:00.908271 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:00.909799 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:00.910300 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:00.910303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:00.910305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:00.910398 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:00.910708 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:00.910711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:00.910713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:00.910961 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:00.911077 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:00.911607 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 18:40:00.911777 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:40:00.912778 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:00.899109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:35879 (local address 127.8.241.194:35459)
0806 18:40:00.899605 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:00.899652 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:00.900980 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:00.901652 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:00.901663 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:00.902205 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:00.902844 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:00.902861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.903802 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.903805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:00.905158 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:00.905172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:00.905818 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:00.905821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:00.905987 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:00.906572 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:00.906590 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:00.908030 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:00.909914 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:00.909920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:00.909935 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:00.910203 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:00.910484 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:00.910486 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:00.910487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:00.910623 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:00.911063 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:00.911068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:00.911334 (+ 266us) client_negotiation.cc:769] Sending connection context
0806 18:40:00.911523 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 18:40:00.911814 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":122,"threads_started":1}
I20260806 18:40:00.914067 15011 heartbeater.cc:347] Connected to a master server at 127.8.241.254:35879
I20260806 18:40:00.914322 15011 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:00.914899 15011 heartbeater.cc:511] Master 127.8.241.254:35879 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
I20260806 18:40:00.916082 14518 ts_manager.cc:194] Registered new tserver with Master: 64e243e7a26e48c0a5ee76255f8bea79 (127.8.241.194:46283)
I20260806 18:40:00.916679 14518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:35459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:40:00.930284 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35879
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:01.050360 15018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:01.050653 15018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:01.050704 15018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:01.054942 15018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:01.055033 15018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:01.055109 15018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:01.060650 15018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:01.061875 15018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:01.062989 15018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:01.073377 15024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:01.073371 15027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:01.073375 15025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:01.073647 15018 server_base.cc:1034] running on GCE node
I20260806 18:40:01.074240 15018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:01.074978 15018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:01.076185 15018 hybrid_clock.cc:648] HybridClock initialized: now 1786041601076159 us; error 36 us; skew 500 ppm
Aug 06 18:40:01 dist-test-slave-k5rr krb5kdc[14452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041601, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:01.079639 15018 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:40:01.080681 15018 webserver.cc:460] Webserver started at http://127.8.241.195:36789/ using document root <none> and password file <none>
I20260806 18:40:01.081354 15018 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:01.081427 15018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:01.081605 15018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:01.083761 15018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "3e2ca19629e146f4ad8a5842c42e7070"
format_stamp: "Formatted at 2026-08-06 18:40:01 on dist-test-slave-k5rr"
server_key: "98c04dc693fe4298b067ed6ce2a6657e"
server_key_iv: "4261769d7350b3e863ee089d371e2b54"
server_key_version: "encryptionkey@0"
I20260806 18:40:01.084331 15018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "3e2ca19629e146f4ad8a5842c42e7070"
format_stamp: "Formatted at 2026-08-06 18:40:01 on dist-test-slave-k5rr"
server_key: "98c04dc693fe4298b067ed6ce2a6657e"
server_key_iv: "4261769d7350b3e863ee089d371e2b54"
server_key_version: "encryptionkey@0"
I20260806 18:40:01.088344 15018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:40:01.091087 15045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:01.092451 15018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:40:01.092571 15018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "3e2ca19629e146f4ad8a5842c42e7070"
format_stamp: "Formatted at 2026-08-06 18:40:01 on dist-test-slave-k5rr"
server_key: "98c04dc693fe4298b067ed6ce2a6657e"
server_key_iv: "4261769d7350b3e863ee089d371e2b54"
server_key_version: "encryptionkey@0"
I20260806 18:40:01.092689 15018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:01.105214 15018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:01.108647 15018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:01.108861 15018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:01.109550 15018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:01.110486 15018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:01.110538 15018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:01.110589 15018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:01.110618 15018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:01.119156 15018 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:43283
I20260806 18:40:01.119208 15218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:43283 every 8 connection(s)
I20260806 18:40:01.120088 15018 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:01.128034 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 15018
I20260806 18:40:01.128156 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:01.128425 9159 external_mini_cluster.cc:1428] Setting key b2ea67ecb9d468b29a4dc746c88c4f54
Aug 06 18:40:01 dist-test-slave-k5rr krb5kdc[14452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041601, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:01.133594 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:01.121970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:32877 (local address 127.8.241.254:35879)
0806 18:40:01.122143 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:01.122148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:01.123010 (+ 862us) server_negotiation.cc:407] Connection header received
0806 18:40:01.124361 (+ 1351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:01.124363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:01.124414 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:01.124489 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:01.125671 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:01.126192 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:01.127260 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:01.127473 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:01.127548 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:01.130290 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:01.130294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:01.130297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:01.130325 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:01.131656 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:01.132098 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:01.132101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:01.132102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:01.132170 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:01.132486 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:01.132490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:01.132491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:01.132713 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:01.132803 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:01.133360 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 18:40:01.133473 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:40:01.134534 15223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:01.122373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:35879 (local address 127.8.241.195:32877)
0806 18:40:01.122851 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:01.122896 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:01.124165 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:01.124651 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:01.124658 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:01.125043 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:01.125556 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:01.125570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:01.126279 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:01.126281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:01.127160 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:01.127169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:01.127600 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:01.127603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:01.127746 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:01.128312 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:01.128329 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:01.130133 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:01.131742 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:01.131747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:01.131760 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:01.132031 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:01.132236 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:01.132238 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:01.132239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:01.132373 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:01.132816 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:01.132821 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:01.133125 (+ 304us) client_negotiation.cc:769] Sending connection context
0806 18:40:01.133306 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:40:01.133593 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":112,"threads_started":1}
I20260806 18:40:01.135689 15220 heartbeater.cc:347] Connected to a master server at 127.8.241.254:35879
I20260806 18:40:01.135913 15220 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:01.136428 15220 heartbeater.cc:511] Master 127.8.241.254:35879 requested a full tablet report, sending...
I20260806 18:40:01.137508 14518 ts_manager.cc:194] Registered new tserver with Master: 3e2ca19629e146f4ad8a5842c42e7070 (127.8.241.195:43283)
I20260806 18:40:01.138104 14518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:32877
I20260806 18:40:01.142508 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:40:01 dist-test-slave-k5rr krb5kdc[14452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:40:01 dist-test-slave-k5rr krb5kdc[14452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:40:01.169441 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:01.159779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43304 (local address 127.8.241.193:40589)
0806 18:40:01.160093 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:01.160112 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:01.160139 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:40:01.160197 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:01.160202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:01.160404 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:01.160544 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:01.161292 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:01.162091 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:01.162957 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:01.163354 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:01.163456 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:01.166032 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:01.166044 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:01.166062 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:01.166093 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:01.167779 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:01.168218 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:01.168223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:01.168224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:01.168305 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:01.168506 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:01.168508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:01.168509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:01.168765 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:01.168964 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:01.169137 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 18:40:01.169276 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":68,"threads_started":1}
Aug 06 18:40:01 dist-test-slave-k5rr krb5kdc[14452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:01.179801 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:01.172454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51036 (local address 127.8.241.254:35879)
0806 18:40:01.172607 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:01.172610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:01.172624 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:01.172716 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:01.172719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:01.172763 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:01.172824 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:01.173444 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:01.173970 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:01.174661 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:01.174856 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:01.174919 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:01.177187 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:01.177191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:01.177193 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:01.177219 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:01.178322 (+ 1103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:01.178712 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:01.178715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:01.178717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:01.178796 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:01.179004 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:01.179006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:01.179008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:01.179249 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:01.179349 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:01.179522 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 18:40:01.179618 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 18:40:01.180276 14518 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51036
I20260806 18:40:01.180895 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 14621
I20260806 18:40:01.193257 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 14825
I20260806 18:40:01.208060 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 15018
I20260806 18:40:01.218737 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 14469
2026-08-06T18:40:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2347 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:40:01 dist-test-slave-k5rr krb5kdc[15237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:40:01 dist-test-slave-k5rr krb5kdc[15237](info): set up 2 sockets
Aug 06 18:40:01 dist-test-slave-k5rr krb5kdc[15237](info): commencing operation
krb5kdc: starting...
W20260806 18:40:02.374986 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:40:02 dist-test-slave-k5rr krb5kdc[15237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:40:02.500248 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:35607
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:35607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:02.617818 15258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:02.618155 15258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:02.618220 15258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:02.622417 15258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:02.622499 15258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:02.622527 15258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:02.622548 15258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:02.622574 15258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:02.628196 15258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:35607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:35607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:02.629492 15258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:02.630614 15258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:02.639444 15267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:02.639515 15258 server_base.cc:1034] running on GCE node
W20260806 18:40:02.639472 15265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:02.639472 15264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:02.640336 15258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:02.641661 15258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:02.642838 15258 hybrid_clock.cc:648] HybridClock initialized: now 1786041602642817 us; error 37 us; skew 500 ppm
Aug 06 18:40:02 dist-test-slave-k5rr krb5kdc[15237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041602, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:02.645718 15258 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:40:02.646555 15258 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:02.646626 15258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:02.646848 15258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:02.649200 15258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "c95d2b1bc1054fc89c4d88685d54cea6"
format_stamp: "Formatted at 2026-08-06 18:40:02 on dist-test-slave-k5rr"
server_key: "4a05dc89e594a2a6a501129284ba65bc"
server_key_iv: "a44745b7bc58819d3087f2173b028f9e"
server_key_version: "encryptionkey@0"
I20260806 18:40:02.649824 15258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "c95d2b1bc1054fc89c4d88685d54cea6"
format_stamp: "Formatted at 2026-08-06 18:40:02 on dist-test-slave-k5rr"
server_key: "4a05dc89e594a2a6a501129284ba65bc"
server_key_iv: "a44745b7bc58819d3087f2173b028f9e"
server_key_version: "encryptionkey@0"
I20260806 18:40:02.653944 15258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 18:40:02.656605 15280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:02.658046 15258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:40:02.658180 15258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "c95d2b1bc1054fc89c4d88685d54cea6"
format_stamp: "Formatted at 2026-08-06 18:40:02 on dist-test-slave-k5rr"
server_key: "4a05dc89e594a2a6a501129284ba65bc"
server_key_iv: "a44745b7bc58819d3087f2173b028f9e"
server_key_version: "encryptionkey@0"
I20260806 18:40:02.658290 15258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:02.669932 15258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:02.673324 15258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:02.673524 15258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:02.680507 15258 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:35607
I20260806 18:40:02.680787 15353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:35607 every 8 connection(s)
I20260806 18:40:02.681277 15258 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:02.684294 15354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:02.687758 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 15258
I20260806 18:40:02.687871 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:02.688138 9159 external_mini_cluster.cc:1428] Setting key 602ff6a3cfbe888c8f2b38b8ae904f96
I20260806 18:40:02.690805 15354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6: Bootstrap starting.
I20260806 18:40:02.693871 15354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:02.694808 15354 log.cc:824] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:02.697077 15354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6: No bootstrap required, opened a new log
Aug 06 18:40:02 dist-test-slave-k5rr krb5kdc[15237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:02.701664 15354 raft_consensus.cc:348] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95d2b1bc1054fc89c4d88685d54cea6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35607 } }
I20260806 18:40:02.702009 15354 raft_consensus.cc:374] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:02.702075 15354 raft_consensus.cc:729] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95d2b1bc1054fc89c4d88685d54cea6, State: Initialized, Role: FOLLOWER
I20260806 18:40:02.702627 15354 consensus_queue.cc:260] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95d2b1bc1054fc89c4d88685d54cea6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35607 } }
I20260806 18:40:02.702781 15354 raft_consensus.cc:388] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:02.702850 15354 raft_consensus.cc:482] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:02.702960 15354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:02.703553 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:02.689693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56886 (local address 127.8.241.254:35607)
0806 18:40:02.690149 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:02.690175 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:02.690221 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:40:02.691093 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:02.691118 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:02.691598 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:02.691983 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:02.692674 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:02.694214 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:02.695136 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:02.695712 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:02.695878 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:02.698302 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:02.698309 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:02.698327 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:02.698360 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:02.700270 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:02.700706 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:02.700713 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:02.700720 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:02.700870 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:02.701183 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:02.701186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:02.701188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:02.701744 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:02.701960 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:02.702337 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 18:40:02.702636 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":111,"threads_started":1}
I20260806 18:40:02.704001 15354 raft_consensus.cc:504] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95d2b1bc1054fc89c4d88685d54cea6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35607 } }
I20260806 18:40:02.704385 15354 leader_election.cc:302] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c95d2b1bc1054fc89c4d88685d54cea6; no voters:
I20260806 18:40:02.704763 15354 leader_election.cc:288] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:02.704865 15365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:02.705102 15365 raft_consensus.cc:686] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [term 1 LEADER]: Becoming Leader. State: Replica: c95d2b1bc1054fc89c4d88685d54cea6, State: Running, Role: LEADER
I20260806 18:40:02.705508 15365 consensus_queue.cc:237] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95d2b1bc1054fc89c4d88685d54cea6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35607 } }
I20260806 18:40:02.706099 15354 sys_catalog.cc:564] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:02.706735 15365 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c95d2b1bc1054fc89c4d88685d54cea6. Latest consensus state: current_term: 1 leader_uuid: "c95d2b1bc1054fc89c4d88685d54cea6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95d2b1bc1054fc89c4d88685d54cea6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35607 } } }
I20260806 18:40:02.706861 15365 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:02.707274 15374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:02.707245 15366 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c95d2b1bc1054fc89c4d88685d54cea6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95d2b1bc1054fc89c4d88685d54cea6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35607 } } }
I20260806 18:40:02.707434 15366 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:02.710891 15374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:02.717418 15374 catalog_manager.cc:1347] Generated new cluster ID: 0b0bb4c3ae1e477db8d3376ac8aa3edd
I20260806 18:40:02.717520 15374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:02.728379 15374 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:02.729434 15374 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:02.740279 15374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6: Generated new TSK 0
I20260806 18:40:02.741098 15374 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:40:02.774559 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35607
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:02.910871 15400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:02.911176 15400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:02.911228 15400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:02.915467 15400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:02.915549 15400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:02.915620 15400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:02.921131 15400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:02.922271 15400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:02.923367 15400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:02.933692 15409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:02.933710 15408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:02.933692 15411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:02.934022 15400 server_base.cc:1034] running on GCE node
I20260806 18:40:02.934509 15400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:02.935235 15400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:02.936395 15400 hybrid_clock.cc:648] HybridClock initialized: now 1786041602936365 us; error 49 us; skew 500 ppm
Aug 06 18:40:02 dist-test-slave-k5rr krb5kdc[15237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041602, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:02.939316 15400 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:40:02.940169 15400 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:02.940253 15400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:02.940503 15400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:02.942833 15400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "4d9edf04bd2641fbb290b8eea6e799dd"
format_stamp: "Formatted at 2026-08-06 18:40:02 on dist-test-slave-k5rr"
server_key: "7a76ef37730bce7f3725f93419d81c80"
server_key_iv: "4908f2e10c4a36d854f0503128329c20"
server_key_version: "encryptionkey@0"
I20260806 18:40:02.943424 15400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "4d9edf04bd2641fbb290b8eea6e799dd"
format_stamp: "Formatted at 2026-08-06 18:40:02 on dist-test-slave-k5rr"
server_key: "7a76ef37730bce7f3725f93419d81c80"
server_key_iv: "4908f2e10c4a36d854f0503128329c20"
server_key_version: "encryptionkey@0"
I20260806 18:40:02.947499 15400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 18:40:02.950204 15418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:02.951565 15400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:40:02.951694 15400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "4d9edf04bd2641fbb290b8eea6e799dd"
format_stamp: "Formatted at 2026-08-06 18:40:02 on dist-test-slave-k5rr"
server_key: "7a76ef37730bce7f3725f93419d81c80"
server_key_iv: "4908f2e10c4a36d854f0503128329c20"
server_key_version: "encryptionkey@0"
I20260806 18:40:02.951803 15400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:02.993938 15400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:02.997057 15400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:02.997259 15400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:02.997915 15400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:02.998855 15400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:02.998907 15400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:02.998966 15400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:02.998996 15400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:03.008064 15400 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:44485
I20260806 18:40:03.008081 15591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:44485 every 8 connection(s)
I20260806 18:40:03.008793 15400 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:03.013439 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 15400
I20260806 18:40:03.013581 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:03.013897 9159 external_mini_cluster.cc:1428] Setting key 505cc51d5921e4551d0fd31e33f236aa
Aug 06 18:40:03 dist-test-slave-k5rr krb5kdc[15237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041602, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:03.024473 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.010661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:58677 (local address 127.8.241.254:35607)
0806 18:40:03.010846 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.010851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.012190 (+ 1339us) server_negotiation.cc:407] Connection header received
0806 18:40:03.013183 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.013186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.013241 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.013313 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:03.014974 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.015679 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.017373 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.017714 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.017885 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.020633 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:03.020637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:03.020640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:03.020674 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:03.022268 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:03.022807 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:03.022811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:03.022813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:03.022908 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:03.023231 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:03.023235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:03.023236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:03.023493 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:03.023601 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.024124 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.024279 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:40:03.025326 15594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.011146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:35607 (local address 127.8.241.193:58677)
0806 18:40:03.012002 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.012049 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:03.012905 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:03.013557 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:03.013568 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:03.014091 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:03.014809 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.014833 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.015820 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.015824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.017245 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.017258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.017921 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.017924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.018086 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.018706 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:03.018724 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:03.020408 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:03.022391 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:03.022397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:03.022414 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:03.022699 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:03.023001 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:03.023003 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:03.023004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:03.023145 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:03.023590 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:03.023595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:03.023866 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 18:40:03.024040 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 18:40:03.024326 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":121,"threads_started":1}
I20260806 18:40:03.026672 15592 heartbeater.cc:347] Connected to a master server at 127.8.241.254:35607
I20260806 18:40:03.026929 15592 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:03.027455 15592 heartbeater.cc:511] Master 127.8.241.254:35607 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:40:03.029170 15305 ts_manager.cc:194] Registered new tserver with Master: 4d9edf04bd2641fbb290b8eea6e799dd (127.8.241.193:44485)
I20260806 18:40:03.030670 15305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:58677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:40:03.044885 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35607
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:03.163908 15599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:03.164220 15599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:03.164273 15599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:03.168551 15599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:03.168637 15599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:03.168710 15599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:03.174271 15599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:03.175443 15599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:03.176545 15599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:03.186430 15608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:03.186451 15610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:03.186616 15599 server_base.cc:1034] running on GCE node
W20260806 18:40:03.186443 15607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:03.187165 15599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:03.187876 15599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:03.189055 15599 hybrid_clock.cc:648] HybridClock initialized: now 1786041603189031 us; error 32 us; skew 500 ppm
Aug 06 18:40:03 dist-test-slave-k5rr krb5kdc[15237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041603, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:03.191901 15599 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:40:03.192698 15599 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:03.192763 15599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:03.192976 15599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:03.195165 15599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "b15d0a705f254acfa5b2bdd116aaa95b"
format_stamp: "Formatted at 2026-08-06 18:40:03 on dist-test-slave-k5rr"
server_key: "aa7d9993bc096e7af2b71932f2e5d9db"
server_key_iv: "dcf823a5c875c22f670a3863a72024e1"
server_key_version: "encryptionkey@0"
I20260806 18:40:03.195724 15599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "b15d0a705f254acfa5b2bdd116aaa95b"
format_stamp: "Formatted at 2026-08-06 18:40:03 on dist-test-slave-k5rr"
server_key: "aa7d9993bc096e7af2b71932f2e5d9db"
server_key_iv: "dcf823a5c875c22f670a3863a72024e1"
server_key_version: "encryptionkey@0"
I20260806 18:40:03.199522 15599 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 18:40:03.202105 15616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:03.203408 15599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:03.203536 15599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "b15d0a705f254acfa5b2bdd116aaa95b"
format_stamp: "Formatted at 2026-08-06 18:40:03 on dist-test-slave-k5rr"
server_key: "aa7d9993bc096e7af2b71932f2e5d9db"
server_key_iv: "dcf823a5c875c22f670a3863a72024e1"
server_key_version: "encryptionkey@0"
I20260806 18:40:03.203650 15599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:03.232334 15599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:03.235615 15599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:03.235813 15599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:03.236524 15599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:03.237537 15599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:03.237591 15599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:03.237649 15599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:03.237681 15599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:03.246883 15599 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:40549
I20260806 18:40:03.247164 15786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:40549 every 8 connection(s)
I20260806 18:40:03.247592 15599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:03.252658 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 15599
I20260806 18:40:03.252759 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:40:03.253073 9159 external_mini_cluster.cc:1428] Setting key 8057b3b996234450d89d3318d8cff3f1
Aug 06 18:40:03 dist-test-slave-k5rr krb5kdc[15237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041603, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:03.262154 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.249442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:50589 (local address 127.8.241.254:35607)
0806 18:40:03.249584 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.249588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.250862 (+ 1274us) server_negotiation.cc:407] Connection header received
0806 18:40:03.251810 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.251814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.251875 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.251978 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:03.253283 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.253817 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.254990 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.255301 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.255395 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.258288 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:03.258292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:03.258294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:03.258321 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:03.259758 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:03.260397 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:03.260399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:03.260401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:03.260473 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:03.260859 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:03.260861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:03.260862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:03.261095 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:03.261174 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.261886 (+ 712us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.262020 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:40:03.263531 15790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.249856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:35607 (local address 127.8.241.194:50589)
0806 18:40:03.250706 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.250750 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:03.251600 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:03.252140 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:03.252149 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:03.252537 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:03.253130 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.253153 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.253947 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.253951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.254859 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.254870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.255471 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.255477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.255649 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.256287 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:03.256312 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:03.258091 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:03.259891 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:03.259899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:03.259922 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:03.260304 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:03.260554 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:03.260558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:03.260560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:03.260769 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:03.261194 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:03.261203 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:03.261621 (+ 418us) client_negotiation.cc:769] Sending connection context
0806 18:40:03.261863 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 18:40:03.262266 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:40:03.264784 15787 heartbeater.cc:347] Connected to a master server at 127.8.241.254:35607
I20260806 18:40:03.265120 15787 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:03.265633 15787 heartbeater.cc:511] Master 127.8.241.254:35607 requested a full tablet report, sending...
I20260806 18:40:03.266803 15305 ts_manager.cc:194] Registered new tserver with Master: b15d0a705f254acfa5b2bdd116aaa95b (127.8.241.194:40549)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
I20260806 18:40:03.267514 15305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:50589
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:40:03.282245 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35607
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:03.399961 15794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:03.400247 15794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:03.400292 15794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:03.404309 15794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:03.404381 15794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:03.404441 15794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:03.410593 15794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:35607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:03.411780 15794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:03.412855 15794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:03.422804 15804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:03.422879 15803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:03.422989 15794 server_base.cc:1034] running on GCE node
W20260806 18:40:03.422817 15807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:03.423544 15794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:03.424299 15794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:03.425462 15794 hybrid_clock.cc:648] HybridClock initialized: now 1786041603425438 us; error 37 us; skew 500 ppm
Aug 06 18:40:03 dist-test-slave-k5rr krb5kdc[15237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041603, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:03.428298 15794 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:40:03.429140 15794 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:03.429219 15794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:03.429435 15794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:03.431600 15794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "ee884878975d4a7c9558a2391ee5f6e1"
format_stamp: "Formatted at 2026-08-06 18:40:03 on dist-test-slave-k5rr"
server_key: "1acef5e9cc94a22ce8dc4fc39e482463"
server_key_iv: "3940ef4496652f5dbc7fb7b9ed178b12"
server_key_version: "encryptionkey@0"
I20260806 18:40:03.432173 15794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "ee884878975d4a7c9558a2391ee5f6e1"
format_stamp: "Formatted at 2026-08-06 18:40:03 on dist-test-slave-k5rr"
server_key: "1acef5e9cc94a22ce8dc4fc39e482463"
server_key_iv: "3940ef4496652f5dbc7fb7b9ed178b12"
server_key_version: "encryptionkey@0"
I20260806 18:40:03.435997 15794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:40:03.438666 15814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:03.439960 15794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:40:03.440083 15794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "ee884878975d4a7c9558a2391ee5f6e1"
format_stamp: "Formatted at 2026-08-06 18:40:03 on dist-test-slave-k5rr"
server_key: "1acef5e9cc94a22ce8dc4fc39e482463"
server_key_iv: "3940ef4496652f5dbc7fb7b9ed178b12"
server_key_version: "encryptionkey@0"
I20260806 18:40:03.440186 15794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:03.457660 15794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:03.460973 15794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:03.461198 15794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:03.461825 15794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:03.462821 15794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:03.462870 15794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:03.462926 15794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:03.462958 15794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:03.471572 15794 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:39415
I20260806 18:40:03.471589 16004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:39415 every 8 connection(s)
I20260806 18:40:03.472222 15794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:03.479472 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 15794
I20260806 18:40:03.479612 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:03.479918 9159 external_mini_cluster.cc:1428] Setting key 30e4dfc3e6be8806c2f665e9b4620e49
Aug 06 18:40:03 dist-test-slave-k5rr krb5kdc[15237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041603, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:03.487795 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.474153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:48125 (local address 127.8.241.254:35607)
0806 18:40:03.474290 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.474294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.475531 (+ 1237us) server_negotiation.cc:407] Connection header received
0806 18:40:03.476426 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.476430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.476488 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.476567 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:03.478115 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.478865 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.480483 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.480905 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.481041 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.484241 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:03.484246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:03.484248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:03.484279 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:03.485727 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:03.486240 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:03.486244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:03.486246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:03.486329 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:03.486653 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:03.486657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:03.486659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:03.486856 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:03.486947 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.487489 (+ 542us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.487624 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:40:03.488672 16007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.474543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:35607 (local address 127.8.241.195:48125)
0806 18:40:03.475371 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.475414 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:03.476226 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:03.476771 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:03.476778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:03.477283 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:03.477958 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.477977 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.479003 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.479007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.480288 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.480301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.481225 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.481230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.481453 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.482114 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:03.482136 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:03.484058 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:03.485842 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:03.485849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:03.485867 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:03.486143 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:03.486412 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:03.486416 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:03.486418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:03.486562 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:03.486967 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:03.486975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:03.487243 (+ 268us) client_negotiation.cc:769] Sending connection context
0806 18:40:03.487422 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 18:40:03.487775 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":118,"threads_started":1}
I20260806 18:40:03.489933 16005 heartbeater.cc:347] Connected to a master server at 127.8.241.254:35607
I20260806 18:40:03.490217 16005 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:03.490715 16005 heartbeater.cc:511] Master 127.8.241.254:35607 requested a full tablet report, sending...
I20260806 18:40:03.491819 15305 ts_manager.cc:194] Registered new tserver with Master: ee884878975d4a7c9558a2391ee5f6e1 (127.8.241.195:39415)
I20260806 18:40:03.492384 15305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:48125
I20260806 18:40:03.494695 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:40:03.503372 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.496621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56902 (local address 127.8.241.254:35607)
0806 18:40:03.496759 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.496763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.496927 (+ 164us) server_negotiation.cc:407] Connection header received
0806 18:40:03.497098 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.497101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.497148 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.497221 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:03.497890 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.498416 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.499213 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.499447 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.499529 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.500799 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:03.500802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:03.500804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:03.500827 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:03.502057 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:03.502429 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:03.502432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:03.502433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:03.502500 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:03.502680 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:03.502681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:03.502683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:03.502887 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:03.502987 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.503147 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.503248 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:40:03.506356 15305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:40:03.509707 15305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:03.524761 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.518427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:40549 (local address 127.0.0.1:59082)
0806 18:40:03.518905 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.518965 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:03.519107 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:03.519958 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:03.519966 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:03.519993 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:03.520157 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.520164 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.521854 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.521857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.523373 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.523380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.524176 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.524180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.524294 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.524338 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 18:40:03.524461 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 18:40:03.524614 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":113,"thread_start_us":98,"threads_started":1}
I20260806 18:40:03.524767 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.518596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.8.241.194:40549)
0806 18:40:03.519401 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.519414 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.519438 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:40:03.519536 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.519542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.519765 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.520315 (+ 550us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:03.520490 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.521698 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.523502 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.524058 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.524162 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.524327 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.524498 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.524597 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":157,"threads_started":1}
I20260806 18:40:03.525074 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.519373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.8.241.195:39415)
0806 18:40:03.520014 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.520026 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.520048 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:40:03.520123 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.520128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.520330 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.520758 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:03.520914 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.521987 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.523322 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.524157 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.524590 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.524732 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.524782 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.524872 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":153,"threads_started":1}
I20260806 18:40:03.525319 16021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.518483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:39415 (local address 127.0.0.1:39832)
0806 18:40:03.519031 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.519044 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:03.519107 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:03.520485 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:03.520489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:03.520509 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:03.520611 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.520617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.522108 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.522111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.523189 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.523199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.524306 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.524310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.524411 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.524434 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:40:03.524495 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 18:40:03.525181 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":611,"thread_start_us":91,"threads_started":1}
I20260806 18:40:03.525336 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.519239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44485 (local address 127.0.0.1:36756)
0806 18:40:03.519495 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.519508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:03.519602 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:03.520721 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:03.520724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:03.520739 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:03.520820 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.520824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.521829 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.521833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.523355 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.523368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.524583 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.524586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.524675 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.524695 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:40:03.524759 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:40:03.524828 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":70,"thread_start_us":101,"threads_started":1}
I20260806 18:40:03.525668 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.519700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.8.241.193:44485)
0806 18:40:03.520086 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.520092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.520116 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:40:03.520191 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.520196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.520398 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.520567 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:03.520935 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.521706 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.523492 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.524276 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.524367 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.524523 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.525062 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.525522 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":104,"threads_started":1}
I20260806 18:40:03.528209 15686 tablet_service.cc:1467] Processing CreateTablet for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd (DEFAULT_TABLE table=test-table [id=5322a169cfc148a883a5c3cd0580054c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:03.528586 15896 tablet_service.cc:1467] Processing CreateTablet for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd (DEFAULT_TABLE table=test-table [id=5322a169cfc148a883a5c3cd0580054c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:03.528586 15492 tablet_service.cc:1467] Processing CreateTablet for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd (DEFAULT_TABLE table=test-table [id=5322a169cfc148a883a5c3cd0580054c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:03.529345 15686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:03.529709 15896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:03.529701 15492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:03.538053 16029 tablet_bootstrap.cc:492] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd: Bootstrap starting.
I20260806 18:40:03.538169 16030 tablet_bootstrap.cc:492] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: Bootstrap starting.
I20260806 18:40:03.538508 16028 tablet_bootstrap.cc:492] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b: Bootstrap starting.
I20260806 18:40:03.540455 16028 tablet_bootstrap.cc:654] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:03.540544 16029 tablet_bootstrap.cc:654] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:03.540544 16030 tablet_bootstrap.cc:654] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:03.541535 16028 log.cc:824] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:03.541543 16029 log.cc:824] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:03.541652 16030 log.cc:824] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:03.543802 16028 tablet_bootstrap.cc:492] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b: No bootstrap required, opened a new log
I20260806 18:40:03.543814 16030 tablet_bootstrap.cc:492] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: No bootstrap required, opened a new log
I20260806 18:40:03.543815 16029 tablet_bootstrap.cc:492] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd: No bootstrap required, opened a new log
I20260806 18:40:03.544034 16029 ts_tablet_manager.cc:1397] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:40:03.544034 16030 ts_tablet_manager.cc:1397] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 18:40:03.544109 16028 ts_tablet_manager.cc:1397] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:40:03.547200 16028 raft_consensus.cc:348] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } }
I20260806 18:40:03.547487 16028 raft_consensus.cc:374] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:03.547545 16028 raft_consensus.cc:729] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15d0a705f254acfa5b2bdd116aaa95b, State: Initialized, Role: FOLLOWER
I20260806 18:40:03.548051 16028 consensus_queue.cc:260] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } }
I20260806 18:40:03.547967 16030 raft_consensus.cc:348] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } }
I20260806 18:40:03.548265 16030 raft_consensus.cc:374] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:03.548331 16030 raft_consensus.cc:729] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee884878975d4a7c9558a2391ee5f6e1, State: Initialized, Role: FOLLOWER
I20260806 18:40:03.548456 16029 raft_consensus.cc:348] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } }
I20260806 18:40:03.548813 16030 consensus_queue.cc:260] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } }
I20260806 18:40:03.548938 15787 heartbeater.cc:503] Master 127.8.241.254:35607 was elected leader, sending a full tablet report...
I20260806 18:40:03.549206 16028 ts_tablet_manager.cc:1428] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:40:03.549572 16005 heartbeater.cc:503] Master 127.8.241.254:35607 was elected leader, sending a full tablet report...
I20260806 18:40:03.549835 16030 ts_tablet_manager.cc:1428] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:40:03.548723 16029 raft_consensus.cc:374] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:03.550148 16029 raft_consensus.cc:729] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9edf04bd2641fbb290b8eea6e799dd, State: Initialized, Role: FOLLOWER
I20260806 18:40:03.550624 16029 consensus_queue.cc:260] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } }
I20260806 18:40:03.551318 15592 heartbeater.cc:503] Master 127.8.241.254:35607 was elected leader, sending a full tablet report...
I20260806 18:40:03.551618 16029 ts_tablet_manager.cc:1428] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 18:40:03.584985 16038 raft_consensus.cc:482] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:03.585266 16038 raft_consensus.cc:504] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } }
I20260806 18:40:03.586470 16038 leader_election.cc:288] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b15d0a705f254acfa5b2bdd116aaa95b (127.8.241.194:40549), ee884878975d4a7c9558a2391ee5f6e1 (127.8.241.195:39415)
I20260806 18:40:03.590416 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.586841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:37297 (local address 127.8.241.194:40549)
0806 18:40:03.586994 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.586998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.587170 (+ 172us) server_negotiation.cc:407] Connection header received
0806 18:40:03.587307 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.587311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.587354 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.587416 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:03.587702 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.588202 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.589443 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.589832 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.589951 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.590012 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.590216 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.590287 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:40:03.590447 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.586860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:45573 (local address 127.8.241.195:39415)
0806 18:40:03.587129 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.587134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.587237 (+ 103us) server_negotiation.cc:407] Connection header received
0806 18:40:03.587367 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.587370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.587412 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.587475 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:03.587769 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.588262 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.589445 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.589844 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.589961 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.590018 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.590246 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.590321 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260806 18:40:03.590463 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.586723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:40549 (local address 127.8.241.193:37297)
0806 18:40:03.587065 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.587079 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:03.587204 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:03.587446 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:03.587449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:03.587465 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:03.587596 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.587602 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.588310 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.588313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.589323 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.589334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.589950 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.589953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.590088 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.590120 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 18:40:03.590186 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:40:03.590263 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":83,"thread_start_us":148,"threads_started":1}
I20260806 18:40:03.590477 16041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.586753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:39415 (local address 127.8.241.193:45573)
0806 18:40:03.587149 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.587162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:03.587269 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:03.587507 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:03.587510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:03.587526 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:03.587667 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.587673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.588373 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.588377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.589327 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:03.589337 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.589955 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.589958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:03.590133 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.590152 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:40:03.590219 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:40:03.590293 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":135,"thread_start_us":125,"threads_started":1}
I20260806 18:40:03.591339 15933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b1024d2b2b42d1ad1f6783b18ec4dd" candidate_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee884878975d4a7c9558a2391ee5f6e1" is_pre_election: true
I20260806 18:40:03.591331 15717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b1024d2b2b42d1ad1f6783b18ec4dd" candidate_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" is_pre_election: true
I20260806 18:40:03.591663 15717 raft_consensus.cc:2454] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d9edf04bd2641fbb290b8eea6e799dd in term 0.
I20260806 18:40:03.591665 15933 raft_consensus.cc:2454] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d9edf04bd2641fbb290b8eea6e799dd in term 0.
I20260806 18:40:03.592139 15420 leader_election.cc:302] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9edf04bd2641fbb290b8eea6e799dd, ee884878975d4a7c9558a2391ee5f6e1; no voters:
I20260806 18:40:03.592427 16038 raft_consensus.cc:2781] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:03.592515 16038 raft_consensus.cc:482] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:03.592547 16038 raft_consensus.cc:3037] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:03.593688 16038 raft_consensus.cc:504] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } }
I20260806 18:40:03.594056 16038 leader_election.cc:288] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [CANDIDATE]: Term 1 election: Requested vote from peers b15d0a705f254acfa5b2bdd116aaa95b (127.8.241.194:40549), ee884878975d4a7c9558a2391ee5f6e1 (127.8.241.195:39415)
I20260806 18:40:03.594419 15933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b1024d2b2b42d1ad1f6783b18ec4dd" candidate_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee884878975d4a7c9558a2391ee5f6e1"
I20260806 18:40:03.594419 15717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b1024d2b2b42d1ad1f6783b18ec4dd" candidate_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b15d0a705f254acfa5b2bdd116aaa95b"
I20260806 18:40:03.594558 15717 raft_consensus.cc:3037] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:03.594556 15933 raft_consensus.cc:3037] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:03.595896 15933 raft_consensus.cc:2454] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9edf04bd2641fbb290b8eea6e799dd in term 1.
I20260806 18:40:03.595896 15717 raft_consensus.cc:2454] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9edf04bd2641fbb290b8eea6e799dd in term 1.
I20260806 18:40:03.596257 15420 leader_election.cc:302] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4d9edf04bd2641fbb290b8eea6e799dd, b15d0a705f254acfa5b2bdd116aaa95b, ee884878975d4a7c9558a2391ee5f6e1; no voters:
I20260806 18:40:03.596455 16038 raft_consensus.cc:2781] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:03.596699 16038 raft_consensus.cc:686] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [term 1 LEADER]: Becoming Leader. State: Replica: 4d9edf04bd2641fbb290b8eea6e799dd, State: Running, Role: LEADER
I20260806 18:40:03.597095 16038 consensus_queue.cc:237] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } }
I20260806 18:40:03.600562 15304 catalog_manager.cc:5495] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d9edf04bd2641fbb290b8eea6e799dd (127.8.241.193). New cstate: current_term: 1 leader_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d9edf04bd2641fbb290b8eea6e799dd" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44485 } health_report { overall_health: HEALTHY } } }
I20260806 18:40:03.626336 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.622827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36762 (local address 127.8.241.193:44485)
0806 18:40:03.622976 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.622981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.622997 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:40:03.623088 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.623090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.623146 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.623237 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:03.623494 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.623997 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.624817 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:03.625073 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:03.625169 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:03.625500 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:03.625953 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:03.626069 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.626126 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.626191 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:40:03.632576 15933 raft_consensus.cc:1261] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [term 1 FOLLOWER]: Refusing update from remote peer 4d9edf04bd2641fbb290b8eea6e799dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:03.632575 15717 raft_consensus.cc:1261] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [term 1 FOLLOWER]: Refusing update from remote peer 4d9edf04bd2641fbb290b8eea6e799dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:03.633368 16038 consensus_queue.cc:1035] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee884878975d4a7c9558a2391ee5f6e1" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:03.633581 16042 consensus_queue.cc:1035] T 52b1024d2b2b42d1ad1f6783b18ec4dd P 4d9edf04bd2641fbb290b8eea6e799dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15d0a705f254acfa5b2bdd116aaa95b" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 40549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:03.651060 15302 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:03.651268 15302 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:03.653980 15302 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c95d2b1bc1054fc89c4d88685d54cea6: Sending DeleteTablet for 3 replicas of tablet 52b1024d2b2b42d1ad1f6783b18ec4dd
I20260806 18:40:03.654888 15896 tablet_service.cc:1514] Processing DeleteTablet for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:03 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:39832
I20260806 18:40:03.654914 15686 tablet_service.cc:1514] Processing DeleteTablet for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:03 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:59082
I20260806 18:40:03.655009 15492 tablet_service.cc:1514] Processing DeleteTablet for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:03 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:36756
I20260806 18:40:03.655552 16066 tablet_replica.cc:330] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: stopping tablet replica
I20260806 18:40:03.655671 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 15400
I20260806 18:40:03.655894 16066 raft_consensus.cc:2229] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:03.656217 16066 raft_consensus.cc:2258] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:03.656550 16067 tablet_replica.cc:330] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b: stopping tablet replica
I20260806 18:40:03.656821 16067 raft_consensus.cc:2229] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:03.657080 16067 raft_consensus.cc:2258] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:03.669142 16066 ts_tablet_manager.cc:1905] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:03.671589 16066 ts_tablet_manager.cc:1918] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:03.671715 16066 log.cc:1196] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/52b1024d2b2b42d1ad1f6783b18ec4dd
I20260806 18:40:03.672112 16066 ts_tablet_manager.cc:1939] T 52b1024d2b2b42d1ad1f6783b18ec4dd P ee884878975d4a7c9558a2391ee5f6e1: Deleting consensus metadata
I20260806 18:40:03.673085 16067 ts_tablet_manager.cc:1905] T 52b1024d2b2b42d1ad1f6783b18ec4dd P b15d0a705f254acfa5b2bdd116aaa95b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:03.673687 15282 catalog_manager.cc:4841] TS ee884878975d4a7c9558a2391ee5f6e1 (127.8.241.195:39415): tablet 52b1024d2b2b42d1ad1f6783b18ec4dd (table test-table [id=5322a169cfc148a883a5c3cd0580054c]) successfully deleted
I20260806 18:40:03.674437 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 15599
W20260806 18:40:03.674441 15284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:44485 (error 108)
I20260806 18:40:03.680588 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.680214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44485 (local address 127.0.0.1:36778)
0806 18:40:03.680435 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.680508 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260806 18:40:03.680871 15284 catalog_manager.cc:4584] TS 4d9edf04bd2641fbb290b8eea6e799dd (127.8.241.193:44485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52b1024d2b2b42d1ad1f6783b18ec4dd: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44485: connect: Connection refused (error 111)
I20260806 18:40:03.688884 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 15794
I20260806 18:40:03.691763 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.689084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:40549 (local address 127.0.0.1:59088)
0806 18:40:03.691598 (+ 2514us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:03.691670 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.194:40549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2406}
I20260806 18:40:03.705410 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 15258
2026-08-06T18:40:03Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2485 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T18:40:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:03Z Disabled control of system clock
I20260806 18:40:03.743799 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45475
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:45475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:40:03.859011 16075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:03.859334 16075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:03.859375 16075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:03.863440 16075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:03.863510 16075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:03.863524 16075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:03.863540 16075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:03.863560 16075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:03.868573 16075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:45475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:03.869851 16075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:03.871248 16075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:03.880285 16086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:03.880301 16082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:03.880285 16089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:03.880535 16075 server_base.cc:1034] running on GCE node
I20260806 18:40:03.881340 16075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:03.882566 16075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:03.883728 16075 hybrid_clock.cc:648] HybridClock initialized: now 1786041603883697 us; error 49 us; skew 500 ppm
I20260806 18:40:03.885797 16075 webserver.cc:460] Webserver started at http://127.8.241.254:33317/ using document root <none> and password file <none>
I20260806 18:40:03.886420 16075 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:03.886500 16075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:03.886715 16075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:03.888926 16075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "ef24fa01f3db4a5e86c220bc21daaf9d"
format_stamp: "Formatted at 2026-08-06 18:40:03 on dist-test-slave-k5rr"
server_key: "b56f32aa7bd2a1fb469c654c5904507e"
server_key_iv: "3f0b4edd62592a161ef65d114403e216"
server_key_version: "encryptionkey@0"
I20260806 18:40:03.889575 16075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "ef24fa01f3db4a5e86c220bc21daaf9d"
format_stamp: "Formatted at 2026-08-06 18:40:03 on dist-test-slave-k5rr"
server_key: "b56f32aa7bd2a1fb469c654c5904507e"
server_key_iv: "3f0b4edd62592a161ef65d114403e216"
server_key_version: "encryptionkey@0"
I20260806 18:40:03.893594 16075 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:40:03.896270 16097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:03.897657 16075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:40:03.897794 16075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "ef24fa01f3db4a5e86c220bc21daaf9d"
format_stamp: "Formatted at 2026-08-06 18:40:03 on dist-test-slave-k5rr"
server_key: "b56f32aa7bd2a1fb469c654c5904507e"
server_key_iv: "3f0b4edd62592a161ef65d114403e216"
server_key_version: "encryptionkey@0"
I20260806 18:40:03.897912 16075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:03.902765 16075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:03.903458 16075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:03.903641 16075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:03.910815 16075 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:45475
I20260806 18:40:03.910820 16173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:45475 every 8 connection(s)
I20260806 18:40:03.911875 16075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:03.915504 16174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:03.920212 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 16075
I20260806 18:40:03.920343 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:03.920624 9159 external_mini_cluster.cc:1428] Setting key 9f45188051f88bd16cb64f66732e7a54
I20260806 18:40:03.922221 16174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d: Bootstrap starting.
I20260806 18:40:03.924831 16174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:03.925897 16174 log.cc:824] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:03.926745 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:03.922185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.8.241.254:45475)
0806 18:40:03.922899 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:03.922918 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:03.922966 (+ 48us) server_negotiation.cc:407] Connection header received
0806 18:40:03.923657 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:03.923676 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:03.924094 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:03.924416 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:03.925368 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:03.925374 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:03.925387 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:03.925398 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:03.925460 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:03.925476 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:03.925567 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:03.925705 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 18:40:03.925990 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":161,"threads_started":1}
I20260806 18:40:03.928144 16174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d: No bootstrap required, opened a new log
I20260806 18:40:03.932070 16174 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef24fa01f3db4a5e86c220bc21daaf9d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45475 } }
I20260806 18:40:03.932376 16174 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:03.932442 16174 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef24fa01f3db4a5e86c220bc21daaf9d, State: Initialized, Role: FOLLOWER
I20260806 18:40:03.932875 16174 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef24fa01f3db4a5e86c220bc21daaf9d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45475 } }
I20260806 18:40:03.933039 16174 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:03.933109 16174 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:03.933214 16174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:03.934309 16174 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef24fa01f3db4a5e86c220bc21daaf9d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45475 } }
I20260806 18:40:03.934666 16174 leader_election.cc:302] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef24fa01f3db4a5e86c220bc21daaf9d; no voters:
I20260806 18:40:03.934989 16174 leader_election.cc:288] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:03.935094 16180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:03.935316 16180 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [term 1 LEADER]: Becoming Leader. State: Replica: ef24fa01f3db4a5e86c220bc21daaf9d, State: Running, Role: LEADER
I20260806 18:40:03.935657 16180 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef24fa01f3db4a5e86c220bc21daaf9d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45475 } }
I20260806 18:40:03.936270 16174 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:03.937325 16184 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef24fa01f3db4a5e86c220bc21daaf9d. Latest consensus state: current_term: 1 leader_uuid: "ef24fa01f3db4a5e86c220bc21daaf9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef24fa01f3db4a5e86c220bc21daaf9d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45475 } } }
I20260806 18:40:03.937467 16184 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:03.937744 16182 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef24fa01f3db4a5e86c220bc21daaf9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef24fa01f3db4a5e86c220bc21daaf9d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45475 } } }
I20260806 18:40:03.937837 16182 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:03.937989 16195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:03.941357 16195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:03.946877 16195 catalog_manager.cc:1347] Generated new cluster ID: 953575611fbc4df38a898f497ab37456
I20260806 18:40:03.946960 16195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:03.967253 16195 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:03.967617 16195 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:03.977353 16195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d: Generated new TSK 0
I20260806 18:40:03.978140 16195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:40:03.989213 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45475
--builtin_ntp_servers=127.8.241.212:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:40:04.109843 16213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:04.110155 16213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:04.110198 16213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:04.114318 16213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:04.114391 16213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:04.114462 16213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:04.119433 16213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:04.120638 16213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:04.122205 16213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:04.132014 16219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:04.132053 16218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:04.132071 16221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:04.132078 16213 server_base.cc:1034] running on GCE node
I20260806 18:40:04.132721 16213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:04.133416 16213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:04.134577 16213 hybrid_clock.cc:648] HybridClock initialized: now 1786041604134556 us; error 31 us; skew 500 ppm
I20260806 18:40:04.136448 16213 webserver.cc:460] Webserver started at http://127.8.241.193:38997/ using document root <none> and password file <none>
I20260806 18:40:04.137164 16213 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:04.137246 16213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:04.137519 16213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:04.139745 16213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "b343683fed8148df835b6ef9d97a1914"
format_stamp: "Formatted at 2026-08-06 18:40:04 on dist-test-slave-k5rr"
server_key: "3737a8af9d4d60c3ba83998ed3cca446"
server_key_iv: "5d995d1858f8fa27f08e6052b5729e90"
server_key_version: "encryptionkey@0"
I20260806 18:40:04.140331 16213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "b343683fed8148df835b6ef9d97a1914"
format_stamp: "Formatted at 2026-08-06 18:40:04 on dist-test-slave-k5rr"
server_key: "3737a8af9d4d60c3ba83998ed3cca446"
server_key_iv: "5d995d1858f8fa27f08e6052b5729e90"
server_key_version: "encryptionkey@0"
I20260806 18:40:04.144145 16213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:40:04.146776 16232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:04.148007 16213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:04.148124 16213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "b343683fed8148df835b6ef9d97a1914"
format_stamp: "Formatted at 2026-08-06 18:40:04 on dist-test-slave-k5rr"
server_key: "3737a8af9d4d60c3ba83998ed3cca446"
server_key_iv: "5d995d1858f8fa27f08e6052b5729e90"
server_key_version: "encryptionkey@0"
I20260806 18:40:04.148231 16213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:04.152976 16213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:04.153647 16213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:04.153820 16213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:04.154392 16213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:04.155447 16213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:04.155509 16213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:04.155570 16213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:04.155597 16213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:04.164098 16213 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:36095
I20260806 18:40:04.164120 16411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:36095 every 8 connection(s)
I20260806 18:40:04.165220 16213 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:04.166329 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 16213
I20260806 18:40:04.166426 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:04.166680 9159 external_mini_cluster.cc:1428] Setting key 1d1d8285b7674ae990a9b3a4f9e68e6c
I20260806 18:40:04.168745 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45475
--builtin_ntp_servers=127.8.241.212:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 18:40:04.171878 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.167204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:55935 (local address 127.8.241.254:45475)
0806 18:40:04.167342 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.167345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.168589 (+ 1244us) server_negotiation.cc:407] Connection header received
0806 18:40:04.170197 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.170201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.170244 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.170325 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.171427 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.171431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.171433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.171446 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.171477 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.171486 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.171565 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.171693 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.171739 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:40:04.172821 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.167680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45475 (local address 127.8.241.193:55935)
0806 18:40:04.168450 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:04.168494 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:04.169986 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:04.170494 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:04.170501 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:04.170594 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:04.171169 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:04.171170 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:04.171211 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:04.171286 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:04.171558 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:04.171567 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:04.171573 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:40:04.171736 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 18:40:04.172042 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":161,"threads_started":1}
I20260806 18:40:04.174114 16412 heartbeater.cc:347] Connected to a master server at 127.8.241.254:45475
I20260806 18:40:04.174367 16412 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:04.174803 16412 heartbeater.cc:511] Master 127.8.241.254:45475 requested a full tablet report, sending...
I20260806 18:40:04.176618 16119 ts_manager.cc:194] Registered new tserver with Master: b343683fed8148df835b6ef9d97a1914 (127.8.241.193:36095)
W20260806 18:40:04.285863 16417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:04.286164 16417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:04.286206 16417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:04.290357 16417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:04.290433 16417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:04.290500 16417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:04.295504 16417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:04.296684 16417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:04.298198 16417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:04.308116 16429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:04.308108 16426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:04.308115 16427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:04.308337 16417 server_base.cc:1034] running on GCE node
I20260806 18:40:04.308827 16417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:04.309532 16417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:04.310696 16417 hybrid_clock.cc:648] HybridClock initialized: now 1786041604310668 us; error 39 us; skew 500 ppm
I20260806 18:40:04.312602 16417 webserver.cc:460] Webserver started at http://127.8.241.194:40915/ using document root <none> and password file <none>
I20260806 18:40:04.313246 16417 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:04.313320 16417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:04.313534 16417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:04.315682 16417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "d40078e544f1474e9c2e27c8cf7b7219"
format_stamp: "Formatted at 2026-08-06 18:40:04 on dist-test-slave-k5rr"
server_key: "f90acbcfb3ce22303b8b735eed861120"
server_key_iv: "77a3786e3567d362e2b20fc15190fa24"
server_key_version: "encryptionkey@0"
I20260806 18:40:04.316284 16417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "d40078e544f1474e9c2e27c8cf7b7219"
format_stamp: "Formatted at 2026-08-06 18:40:04 on dist-test-slave-k5rr"
server_key: "f90acbcfb3ce22303b8b735eed861120"
server_key_iv: "77a3786e3567d362e2b20fc15190fa24"
server_key_version: "encryptionkey@0"
I20260806 18:40:04.320204 16417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:40:04.322783 16438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:04.324028 16417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:04.324144 16417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "d40078e544f1474e9c2e27c8cf7b7219"
format_stamp: "Formatted at 2026-08-06 18:40:04 on dist-test-slave-k5rr"
server_key: "f90acbcfb3ce22303b8b735eed861120"
server_key_iv: "77a3786e3567d362e2b20fc15190fa24"
server_key_version: "encryptionkey@0"
I20260806 18:40:04.324251 16417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:04.329133 16417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:04.329777 16417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:04.329947 16417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:04.330525 16417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:04.331502 16417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:04.331552 16417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:04.331604 16417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:04.331632 16417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:04.340830 16417 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:46479
I20260806 18:40:04.340849 16602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:46479 every 8 connection(s)
I20260806 18:40:04.341847 16417 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:04.345613 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 16417
I20260806 18:40:04.345736 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:40:04.346060 9159 external_mini_cluster.cc:1428] Setting key d320e1e599e4081a11a15974c7ac3b0a
I20260806 18:40:04.348284 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45475
--builtin_ntp_servers=127.8.241.212:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 18:40:04.348565 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.343747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:44291 (local address 127.8.241.254:45475)
0806 18:40:04.343880 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.343884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.345030 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 18:40:04.346691 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.346694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.346744 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.346828 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.348107 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.348110 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.348112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.348124 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.348146 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.348153 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.348218 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.348380 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.348427 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:40:04.349756 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.344177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45475 (local address 127.8.241.194:44291)
0806 18:40:04.344854 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:04.344894 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:04.346459 (+ 1565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:04.347035 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:04.347043 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:04.347177 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:04.347766 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:04.347768 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:04.347824 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:04.347921 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:04.348231 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:04.348239 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:04.348246 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:40:04.348441 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 18:40:04.348711 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":116,"threads_started":1}
I20260806 18:40:04.351028 16603 heartbeater.cc:347] Connected to a master server at 127.8.241.254:45475
I20260806 18:40:04.351292 16603 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:04.351754 16603 heartbeater.cc:511] Master 127.8.241.254:45475 requested a full tablet report, sending...
I20260806 18:40:04.352839 16119 ts_manager.cc:194] Registered new tserver with Master: d40078e544f1474e9c2e27c8cf7b7219 (127.8.241.194:46479)
W20260806 18:40:04.467990 16606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:04.468287 16606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:04.468346 16606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:04.472482 16606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:04.472563 16606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:04.472646 16606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:04.477829 16606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:04.478997 16606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:04.480463 16606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:04.490707 16612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:04.490697 16617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:04.490772 16606 server_base.cc:1034] running on GCE node
W20260806 18:40:04.490691 16613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:04.491413 16606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:04.492100 16606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:04.493259 16606 hybrid_clock.cc:648] HybridClock initialized: now 1786041604493224 us; error 49 us; skew 500 ppm
I20260806 18:40:04.495236 16606 webserver.cc:460] Webserver started at http://127.8.241.195:45691/ using document root <none> and password file <none>
I20260806 18:40:04.495865 16606 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:04.495929 16606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:04.496148 16606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:04.498349 16606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0"
format_stamp: "Formatted at 2026-08-06 18:40:04 on dist-test-slave-k5rr"
server_key: "9909e82bf781a8c302b1781010e7c81a"
server_key_iv: "e073edd9690249d5a4501c17afcb3f3d"
server_key_version: "encryptionkey@0"
I20260806 18:40:04.498924 16606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0"
format_stamp: "Formatted at 2026-08-06 18:40:04 on dist-test-slave-k5rr"
server_key: "9909e82bf781a8c302b1781010e7c81a"
server_key_iv: "e073edd9690249d5a4501c17afcb3f3d"
server_key_version: "encryptionkey@0"
I20260806 18:40:04.502918 16606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:40:04.505528 16626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:04.506812 16606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:04.506963 16606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0"
format_stamp: "Formatted at 2026-08-06 18:40:04 on dist-test-slave-k5rr"
server_key: "9909e82bf781a8c302b1781010e7c81a"
server_key_iv: "e073edd9690249d5a4501c17afcb3f3d"
server_key_version: "encryptionkey@0"
I20260806 18:40:04.507081 16606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:04.511833 16606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:04.512483 16606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:04.512650 16606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:04.513244 16606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:04.514205 16606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:04.514252 16606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:04.514304 16606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:04.514334 16606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:04.522778 16606 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:38961
I20260806 18:40:04.522797 16815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:38961 every 8 connection(s)
I20260806 18:40:04.523720 16606 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:04.525967 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 16606
I20260806 18:40:04.526077 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:04.526341 9159 external_mini_cluster.cc:1428] Setting key b323c201ddab82e9289b523a3acde230
I20260806 18:40:04.530265 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.525617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:51671 (local address 127.8.241.254:45475)
0806 18:40:04.525770 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.525775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.527071 (+ 1296us) server_negotiation.cc:407] Connection header received
0806 18:40:04.528643 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.528646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.528702 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.528766 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.529889 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.529892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.529895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.529907 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.529927 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.529934 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.530026 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.530119 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.530158 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:40:04.531169 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.526083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45475 (local address 127.8.241.195:51671)
0806 18:40:04.526891 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:04.526959 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:04.528428 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:04.528976 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:04.528984 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:04.529129 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:04.529634 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:04.529635 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:04.529678 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:04.529753 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:04.530006 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:04.530011 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:04.530015 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:40:04.530154 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 18:40:04.530371 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":117,"threads_started":1}
I20260806 18:40:04.532264 16817 heartbeater.cc:347] Connected to a master server at 127.8.241.254:45475
I20260806 18:40:04.532487 16817 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:04.532886 16817 heartbeater.cc:511] Master 127.8.241.254:45475 requested a full tablet report, sending...
I20260806 18:40:04.533950 16119 ts_manager.cc:194] Registered new tserver with Master: 4a1e8b8fbaf249bcaa48aa985705a7d0 (127.8.241.195:38961)
I20260806 18:40:04.540433 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:40:04.543831 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.542443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35382 (local address 127.8.241.254:45475)
0806 18:40:04.542575 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.542579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.542729 (+ 150us) server_negotiation.cc:407] Connection header received
0806 18:40:04.542853 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.542856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.542891 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.542961 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.543565 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.543567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.543569 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.543576 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.543590 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.543595 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.543652 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.543692 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.543723 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:40:04.546859 16119 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:40:04.550057 16119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:04.561936 16830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.558671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:46479 (local address 127.0.0.1:42470)
0806 18:40:04.559016 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:04.559073 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:04.559174 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:04.559951 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:04.559958 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:04.560042 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:04.560620 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:04.560623 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:04.560647 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:04.560725 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:04.561578 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:04.561587 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:04.561594 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:40:04.561709 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 18:40:04.561785 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
I20260806 18:40:04.562063 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.558853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.8.241.194:46479)
0806 18:40:04.559451 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.559463 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.559489 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:40:04.559545 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.559551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.559761 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.560358 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.561391 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.561397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.561409 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.561420 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.561463 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.561497 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.561596 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.561773 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.561869 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":163,"threads_started":1}
I20260806 18:40:04.562459 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.559391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:36095 (local address 127.0.0.1:42568)
0806 18:40:04.559672 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:04.559685 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:04.559757 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:04.560695 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:04.560698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:04.560713 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:04.561180 (+ 467us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:04.561182 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:04.561196 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:04.561230 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:04.561652 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:04.561655 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:04.561658 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:40:04.561722 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:40:04.561792 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260806 18:40:04.562731 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.559495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.8.241.195:38961)
0806 18:40:04.559992 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.560003 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.560027 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:40:04.560079 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.560084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.560263 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.561414 (+ 1151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.562079 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.562085 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.562096 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.562108 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.562149 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.562178 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.562346 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.562445 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.562516 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":123,"threads_started":1}
I20260806 18:40:04.562582 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.558964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:38961 (local address 127.0.0.1:58056)
0806 18:40:04.559281 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:04.559294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:04.559369 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:04.560727 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:04.560730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:04.560745 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:04.561356 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:04.561358 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:04.561369 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:04.561393 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:04.562329 (+ 936us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:04.562333 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:04.562337 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:40:04.562413 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:40:04.562457 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":105,"threads_started":1}
I20260806 18:40:04.563153 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.559692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42568 (local address 127.8.241.193:36095)
0806 18:40:04.560173 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.560184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.560207 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:40:04.560262 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.560267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.560458 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.560592 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.561417 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.561423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.561435 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.561446 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.561491 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.561520 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.561632 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.561868 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.561951 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":155,"threads_started":1}
I20260806 18:40:04.565414 16511 tablet_service.cc:1467] Processing CreateTablet for tablet 1627954ba2744b1c98a885baa134af48 (DEFAULT_TABLE table=test-table [id=6bae7519b96046dcba817ee1145768b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:04.565608 16313 tablet_service.cc:1467] Processing CreateTablet for tablet 1627954ba2744b1c98a885baa134af48 (DEFAULT_TABLE table=test-table [id=6bae7519b96046dcba817ee1145768b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:04.565840 16702 tablet_service.cc:1467] Processing CreateTablet for tablet 1627954ba2744b1c98a885baa134af48 (DEFAULT_TABLE table=test-table [id=6bae7519b96046dcba817ee1145768b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:04.566633 16313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1627954ba2744b1c98a885baa134af48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:04.566635 16511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1627954ba2744b1c98a885baa134af48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:04.566817 16702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1627954ba2744b1c98a885baa134af48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:04.573068 16837 tablet_bootstrap.cc:492] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914: Bootstrap starting.
I20260806 18:40:04.574643 16839 tablet_bootstrap.cc:492] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: Bootstrap starting.
I20260806 18:40:04.575670 16837 tablet_bootstrap.cc:654] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:04.576519 16837 log.cc:824] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:04.576790 16839 tablet_bootstrap.cc:654] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:04.577299 16838 tablet_bootstrap.cc:492] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: Bootstrap starting.
I20260806 18:40:04.577632 16839 log.cc:824] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:04.578795 16837 tablet_bootstrap.cc:492] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914: No bootstrap required, opened a new log
I20260806 18:40:04.579021 16837 ts_tablet_manager.cc:1397] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:40:04.579267 16838 tablet_bootstrap.cc:654] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:04.579761 16839 tablet_bootstrap.cc:492] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: No bootstrap required, opened a new log
I20260806 18:40:04.579962 16839 ts_tablet_manager.cc:1397] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 18:40:04.579957 16838 log.cc:824] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:04.581868 16838 tablet_bootstrap.cc:492] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: No bootstrap required, opened a new log
I20260806 18:40:04.582083 16838 ts_tablet_manager.cc:1397] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:40:04.584017 16837 raft_consensus.cc:348] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } }
I20260806 18:40:04.584336 16837 raft_consensus.cc:374] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:04.584406 16837 raft_consensus.cc:729] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b343683fed8148df835b6ef9d97a1914, State: Initialized, Role: FOLLOWER
I20260806 18:40:04.584472 16839 raft_consensus.cc:348] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } }
I20260806 18:40:04.584726 16839 raft_consensus.cc:374] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:04.584791 16839 raft_consensus.cc:729] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40078e544f1474e9c2e27c8cf7b7219, State: Initialized, Role: FOLLOWER
I20260806 18:40:04.584920 16837 consensus_queue.cc:260] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } }
I20260806 18:40:04.585265 16839 consensus_queue.cc:260] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } }
I20260806 18:40:04.585559 16838 raft_consensus.cc:348] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } }
I20260806 18:40:04.585744 16412 heartbeater.cc:503] Master 127.8.241.254:45475 was elected leader, sending a full tablet report...
I20260806 18:40:04.585793 16838 raft_consensus.cc:374] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:04.585855 16838 raft_consensus.cc:729] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a1e8b8fbaf249bcaa48aa985705a7d0, State: Initialized, Role: FOLLOWER
I20260806 18:40:04.586081 16837 ts_tablet_manager.cc:1428] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 18:40:04.586262 16838 consensus_queue.cc:260] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } }
I20260806 18:40:04.586941 16817 heartbeater.cc:503] Master 127.8.241.254:45475 was elected leader, sending a full tablet report...
I20260806 18:40:04.587146 16838 ts_tablet_manager.cc:1428] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:40:04.587432 16839 ts_tablet_manager.cc:1428] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:40:04.587757 16603 heartbeater.cc:503] Master 127.8.241.254:45475 was elected leader, sending a full tablet report...
W20260806 18:40:04.592790 16604 tablet.cc:2369] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:40:04.666502 16413 tablet.cc:2369] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:40:04.757228 16850 raft_consensus.cc:482] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:04.757475 16850 raft_consensus.cc:504] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } }
I20260806 18:40:04.758659 16850 leader_election.cc:288] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d40078e544f1474e9c2e27c8cf7b7219 (127.8.241.194:46479), b343683fed8148df835b6ef9d97a1914 (127.8.241.193:36095)
I20260806 18:40:04.760533 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.758959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:46479 (local address 127.8.241.195:39973)
0806 18:40:04.759084 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:04.759098 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:04.759206 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:04.759474 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:04.759477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:04.759495 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:04.759984 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:04.759985 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:04.759995 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:04.760024 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:04.760287 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:04.760291 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:04.760294 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:40:04.760376 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:40:04.760415 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 18:40:04.760551 16831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.759074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:39973 (local address 127.8.241.194:46479)
0806 18:40:04.759217 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.759222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.759240 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:40:04.759292 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.759295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.759333 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.759419 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.760154 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.760157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.760160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.760174 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.760199 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.760211 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.760295 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.760373 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.760417 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:40:04.761092 16851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.759259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:36095 (local address 127.8.241.195:57359)
0806 18:40:04.759663 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:04.759674 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:04.759762 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:04.760010 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:04.760013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:04.760028 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:04.760473 (+ 445us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:04.760475 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:04.760485 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:04.760513 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:04.760804 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:04.760807 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:04.760811 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:40:04.760894 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:40:04.760929 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":128,"threads_started":1}
I20260806 18:40:04.761121 16836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.759454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:57359 (local address 127.8.241.193:36095)
0806 18:40:04.759584 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.759588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.759750 (+ 162us) server_negotiation.cc:407] Connection header received
0806 18:40:04.759852 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.759855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.759893 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.759965 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.760650 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.760654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.760656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.760669 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.760691 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.760702 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.760797 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.760913 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.760975 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:40:04.761497 16538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1627954ba2744b1c98a885baa134af48" candidate_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40078e544f1474e9c2e27c8cf7b7219" is_pre_election: true
I20260806 18:40:04.761787 16538 raft_consensus.cc:2454] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a1e8b8fbaf249bcaa48aa985705a7d0 in term 0.
I20260806 18:40:04.762291 16627 leader_election.cc:302] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a1e8b8fbaf249bcaa48aa985705a7d0, d40078e544f1474e9c2e27c8cf7b7219; no voters:
I20260806 18:40:04.762574 16850 raft_consensus.cc:2781] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:04.762684 16850 raft_consensus.cc:482] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:04.762739 16850 raft_consensus.cc:3037] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:04.763078 16343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1627954ba2744b1c98a885baa134af48" candidate_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b343683fed8148df835b6ef9d97a1914" is_pre_election: true
I20260806 18:40:04.763366 16343 raft_consensus.cc:2454] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a1e8b8fbaf249bcaa48aa985705a7d0 in term 0.
I20260806 18:40:04.763958 16850 raft_consensus.cc:504] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } }
I20260806 18:40:04.764297 16850 leader_election.cc:288] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [CANDIDATE]: Term 1 election: Requested vote from peers d40078e544f1474e9c2e27c8cf7b7219 (127.8.241.194:46479), b343683fed8148df835b6ef9d97a1914 (127.8.241.193:36095)
I20260806 18:40:04.764645 16538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1627954ba2744b1c98a885baa134af48" candidate_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40078e544f1474e9c2e27c8cf7b7219"
I20260806 18:40:04.764667 16343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1627954ba2744b1c98a885baa134af48" candidate_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b343683fed8148df835b6ef9d97a1914"
I20260806 18:40:04.764781 16343 raft_consensus.cc:3037] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:04.764782 16538 raft_consensus.cc:3037] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:04.766096 16538 raft_consensus.cc:2454] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a1e8b8fbaf249bcaa48aa985705a7d0 in term 1.
I20260806 18:40:04.766096 16343 raft_consensus.cc:2454] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a1e8b8fbaf249bcaa48aa985705a7d0 in term 1.
I20260806 18:40:04.766407 16627 leader_election.cc:302] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a1e8b8fbaf249bcaa48aa985705a7d0, b343683fed8148df835b6ef9d97a1914; no voters:
I20260806 18:40:04.766585 16850 raft_consensus.cc:2781] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:04.766808 16850 raft_consensus.cc:686] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 1 LEADER]: Becoming Leader. State: Replica: 4a1e8b8fbaf249bcaa48aa985705a7d0, State: Running, Role: LEADER
I20260806 18:40:04.767190 16850 consensus_queue.cc:237] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } }
I20260806 18:40:04.770619 16118 catalog_manager.cc:5495] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a1e8b8fbaf249bcaa48aa985705a7d0 (127.8.241.195). New cstate: current_term: 1 leader_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a1e8b8fbaf249bcaa48aa985705a7d0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38961 } health_report { overall_health: HEALTHY } } }
W20260806 18:40:04.774660 16818 tablet.cc:2369] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:40:04.800554 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.799202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58062 (local address 127.8.241.195:38961)
0806 18:40:04.799337 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:04.799341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:04.799355 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:04.799463 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:04.799466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:04.799535 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:04.799605 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:04.800194 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:04.800197 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:04.800199 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:04.800215 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:04.800234 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:04.800242 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:04.800315 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:04.800369 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 18:40:04.800403 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:40:04.807188 16538 raft_consensus.cc:1261] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [term 1 FOLLOWER]: Refusing update from remote peer 4a1e8b8fbaf249bcaa48aa985705a7d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:04.807189 16343 raft_consensus.cc:1261] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [term 1 FOLLOWER]: Refusing update from remote peer 4a1e8b8fbaf249bcaa48aa985705a7d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:04.807979 16850 consensus_queue.cc:1035] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b343683fed8148df835b6ef9d97a1914" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 36095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:04.808199 16852 consensus_queue.cc:1035] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d40078e544f1474e9c2e27c8cf7b7219" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:04.817075 16856 mvcc.cc:205] Tried to move back new op lower bound from 7315626413281779712 to 7315626413129146368. Current Snapshot: MvccSnapshot[applied={T|T < 7315626413281779712}]
I20260806 18:40:04.826160 16118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:04.826396 16118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:04.829336 16118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef24fa01f3db4a5e86c220bc21daaf9d: Sending DeleteTablet for 3 replicas of tablet 1627954ba2744b1c98a885baa134af48
I20260806 18:40:04.830201 16511 tablet_service.cc:1514] Processing DeleteTablet for tablet 1627954ba2744b1c98a885baa134af48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:04 UTC) from {username='slave'} at 127.0.0.1:42470
I20260806 18:40:04.830331 16313 tablet_service.cc:1514] Processing DeleteTablet for tablet 1627954ba2744b1c98a885baa134af48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:04 UTC) from {username='slave'} at 127.0.0.1:42568
I20260806 18:40:04.830720 16873 tablet_replica.cc:330] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: stopping tablet replica
I20260806 18:40:04.830745 16702 tablet_service.cc:1514] Processing DeleteTablet for tablet 1627954ba2744b1c98a885baa134af48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:04 UTC) from {username='slave'} at 127.0.0.1:58056
I20260806 18:40:04.830855 16875 tablet_replica.cc:330] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914: stopping tablet replica
I20260806 18:40:04.831219 16875 raft_consensus.cc:2229] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:04.831219 16873 raft_consensus.cc:2229] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:04.831637 16875 raft_consensus.cc:2258] T 1627954ba2744b1c98a885baa134af48 P b343683fed8148df835b6ef9d97a1914 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:04.831643 16873 raft_consensus.cc:2258] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:04.831810 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 16213
I20260806 18:40:04.833914 16873 ts_tablet_manager.cc:1905] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:04.837218 16873 ts_tablet_manager.cc:1918] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:04.837335 16873 log.cc:1196] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/1627954ba2744b1c98a885baa134af48
I20260806 18:40:04.837668 16873 ts_tablet_manager.cc:1939] T 1627954ba2744b1c98a885baa134af48 P d40078e544f1474e9c2e27c8cf7b7219: Deleting consensus metadata
I20260806 18:40:04.839197 16099 catalog_manager.cc:4841] TS d40078e544f1474e9c2e27c8cf7b7219 (127.8.241.194:46479): tablet 1627954ba2744b1c98a885baa134af48 (table test-table [id=6bae7519b96046dcba817ee1145768b7]) successfully deleted
I20260806 18:40:04.839273 16876 tablet_replica.cc:330] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: stopping tablet replica
I20260806 18:40:04.839633 16876 raft_consensus.cc:2229] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:40:04.840047 16876 raft_consensus.cc:2258] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:04.841553 16876 ts_tablet_manager.cc:1905] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:04.843910 16876 ts_tablet_manager.cc:1918] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:04.844015 16876 log.cc:1196] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/1627954ba2744b1c98a885baa134af48
I20260806 18:40:04.844304 16876 ts_tablet_manager.cc:1939] T 1627954ba2744b1c98a885baa134af48 P 4a1e8b8fbaf249bcaa48aa985705a7d0: Deleting consensus metadata
I20260806 18:40:04.845523 16100 catalog_manager.cc:4841] TS 4a1e8b8fbaf249bcaa48aa985705a7d0 (127.8.241.195:38961): tablet 1627954ba2744b1c98a885baa134af48 (table test-table [id=6bae7519b96046dcba817ee1145768b7]) successfully deleted
W20260806 18:40:04.846578 16099 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:36095 (error 108)
I20260806 18:40:04.846675 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 16417
I20260806 18:40:04.856667 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:04.856226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:36095 (local address 127.0.0.1:42578)
0806 18:40:04.856465 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:04.856558 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:36095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260806 18:40:04.856971 16099 catalog_manager.cc:4584] TS b343683fed8148df835b6ef9d97a1914 (127.8.241.193:36095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1627954ba2744b1c98a885baa134af48: Network error: Client connection negotiation failed: client connection to 127.8.241.193:36095: connect: Connection refused (error 111)
I20260806 18:40:04.863021 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 16606
I20260806 18:40:04.872473 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 16075
2026-08-06T18:40:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1170 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 18:40:04.905470 9159 mini_oidc.cc:152] Starting JWKS server
I20260806 18:40:04.907334 9159 webserver.cc:460] Webserver started at https://127.0.0.1:40621/ using document root <none> and password file <none>
I20260806 18:40:04.907518 9159 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:40:04.907819 9159 webserver.cc:460] Webserver started at http://127.0.0.1:42987/ using document root <none> and password file <none>
2026-08-06T18:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:04Z Disabled control of system clock
I20260806 18:40:04.918174 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45349
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:45349
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40621/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/testchainca.cert with env {}
W20260806 18:40:05.035045 16889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:05.035356 16889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:05.035414 16889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:05.039698 16889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:05.039791 16889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:05.039814 16889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:05.039835 16889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:05.039855 16889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:40:05.039877 16889 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:40:05.039899 16889 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40621/jwks.json
I20260806 18:40:05.045749 16889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:45349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40621/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:05.046960 16889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:05.048058 16889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:05.057118 16897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:05.057192 16898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:05.057229 16889 server_base.cc:1034] running on GCE node
W20260806 18:40:05.057118 16902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:05.057998 16889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:05.059214 16889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:05.060391 16889 hybrid_clock.cc:648] HybridClock initialized: now 1786041605060373 us; error 43 us; skew 500 ppm
I20260806 18:40:05.062470 16889 webserver.cc:460] Webserver started at http://127.8.241.254:41503/ using document root <none> and password file <none>
I20260806 18:40:05.063092 16889 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:05.063164 16889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:05.063373 16889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:05.065606 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a"
format_stamp: "Formatted at 2026-08-06 18:40:05 on dist-test-slave-k5rr"
server_key: "b11fd5fbba952062eb4e57ebfeffdec0"
server_key_iv: "44441f11bf4b7c9e17688f48ab1c06fd"
server_key_version: "encryptionkey@0"
I20260806 18:40:05.066182 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a"
format_stamp: "Formatted at 2026-08-06 18:40:05 on dist-test-slave-k5rr"
server_key: "b11fd5fbba952062eb4e57ebfeffdec0"
server_key_iv: "44441f11bf4b7c9e17688f48ab1c06fd"
server_key_version: "encryptionkey@0"
I20260806 18:40:05.069985 16889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:40:05.072453 16910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:05.073791 16889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:40:05.073921 16889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a"
format_stamp: "Formatted at 2026-08-06 18:40:05 on dist-test-slave-k5rr"
server_key: "b11fd5fbba952062eb4e57ebfeffdec0"
server_key_iv: "44441f11bf4b7c9e17688f48ab1c06fd"
server_key_version: "encryptionkey@0"
I20260806 18:40:05.074030 16889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:05.088471 16889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:05.089283 16889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:05.089481 16889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:05.096645 16889 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:45349
I20260806 18:40:05.096668 17009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:45349 every 8 connection(s)
I20260806 18:40:05.097774 16889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:05.101724 17010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:05.104760 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 16889
I20260806 18:40:05.104908 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:05.105281 9159 external_mini_cluster.cc:1428] Setting key 9b35ffd190bf0a48c1647dc1d4d5f4ea
I20260806 18:40:05.108737 17010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a: Bootstrap starting.
I20260806 18:40:05.111315 17010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:05.112164 17010 log.cc:824] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:05.114694 17010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a: No bootstrap required, opened a new log
I20260806 18:40:05.115677 17016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:05.106935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.8.241.254:45349)
0806 18:40:05.107714 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:05.107730 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:05.107769 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:40:05.108438 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:05.108469 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:05.108949 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:05.109328 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:05.109813 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.110791 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.111760 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.112252 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.112402 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:05.113369 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:05.113376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:05.113394 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:05.113918 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:05.113999 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:05.114029 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:05.114130 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:05.114326 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 18:40:05.114641 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":151,"threads_started":1}
I20260806 18:40:05.118846 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:40:05.118805 17010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45349 } }
I20260806 18:40:05.119135 17010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:05.119192 17010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53dc1149a3cf45f8b8995eb4f6e88d9a, State: Initialized, Role: FOLLOWER
I20260806 18:40:05.119696 17010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45349 } }
I20260806 18:40:05.119871 17010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:05.119938 17010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:05.120052 17010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:05.121271 17010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45349 } }
I20260806 18:40:05.121596 17010 leader_election.cc:302] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53dc1149a3cf45f8b8995eb4f6e88d9a; no voters:
I20260806 18:40:05.121925 17010 leader_election.cc:288] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:05.122058 17019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:05.122301 17019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [term 1 LEADER]: Becoming Leader. State: Replica: 53dc1149a3cf45f8b8995eb4f6e88d9a, State: Running, Role: LEADER
I20260806 18:40:05.122670 17019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45349 } }
I20260806 18:40:05.123279 17010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:05.124442 17020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53dc1149a3cf45f8b8995eb4f6e88d9a. Latest consensus state: current_term: 1 leader_uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45349 } } }
I20260806 18:40:05.124469 17021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc1149a3cf45f8b8995eb4f6e88d9a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45349 } } }
I20260806 18:40:05.124593 17020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:05.124593 17021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:05.124976 17030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:05.128321 17030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:05.134377 17030 catalog_manager.cc:1347] Generated new cluster ID: 6744fe22dc8146f184498cb00bdb4d3f
I20260806 18:40:05.134459 17030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:05.162302 17030 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:05.163636 17030 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:05.169662 17030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53dc1149a3cf45f8b8995eb4f6e88d9a: Generated new TSK 0
I20260806 18:40:05.170572 17030 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:40:05.198704 17055 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 18:40:05.198892 17055 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:40:05.199347 17016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:05.182225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50014 (local address 127.8.241.254:45349)
0806 18:40:05.182356 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:05.182360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:05.182550 (+ 190us) server_negotiation.cc:407] Connection header received
0806 18:40:05.182782 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:05.182784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:05.182836 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:05.182937 (+ 101us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:40:05.183195 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.183729 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.184954 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.185235 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.185313 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:05.185747 (+ 434us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:40:05.198924 (+ 13177us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:40:05.199227 (+ 303us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 18:40:05.199429 17016 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50014: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 18:40:05.200935 17054 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:05.182079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45349 (local address 127.0.0.1:50014)
0806 18:40:05.182461 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:05.182476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:05.182621 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:05.182960 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:05.182963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:05.182972 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:40:05.183097 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:05.183104 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.183824 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.183827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:05.184829 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:05.184840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.185348 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.185352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:05.185450 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:05.185517 (+ 67us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:40:05.199245 (+ 13728us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:40:05.199525 (+ 280us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.241.254:45349: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":127,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.241.254:45349: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 18:40:05.201848 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 16889
2026-08-06T18:40:05Z chronyd exiting
I20260806 18:40:05.246716 9159 test_util.cc:175] -----------------------------------------------
I20260806 18:40:05.246812 9159 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041577757111-9159-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (343 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 18:40:05.248001 9159 mini_oidc.cc:152] Starting JWKS server
I20260806 18:40:05.248885 9159 webserver.cc:460] Webserver started at https://127.0.0.1:44109/ using document root <none> and password file <none>
I20260806 18:40:05.248972 9159 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:40:05.249260 9159 webserver.cc:460] Webserver started at http://127.0.0.1:42035/ using document root <none> and password file <none>
2026-08-06T18:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:05Z Disabled control of system clock
I20260806 18:40:05.259182 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36341
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:36341
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44109/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/testchainca.cert with env {}
W20260806 18:40:05.375870 17067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:05.376212 17067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:05.376276 17067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:05.380544 17067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:05.380630 17067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:05.380656 17067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:05.380678 17067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:05.380697 17067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:40:05.380719 17067 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:40:05.380746 17067 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44109/jwks.json
I20260806 18:40:05.386546 17067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:36341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44109/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:05.387760 17067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:05.388870 17067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:05.397840 17072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:05.397852 17077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:05.397861 17081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:05.398070 17067 server_base.cc:1034] running on GCE node
I20260806 18:40:05.398808 17067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:05.399982 17067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:05.401157 17067 hybrid_clock.cc:648] HybridClock initialized: now 1786041605401127 us; error 43 us; skew 500 ppm
I20260806 18:40:05.407891 17067 webserver.cc:460] Webserver started at http://127.8.241.254:38693/ using document root <none> and password file <none>
I20260806 18:40:05.408543 17067 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:05.408615 17067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:05.408819 17067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:05.411134 17067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "e8fd929d5f144d9a9e0f272a09f17295"
format_stamp: "Formatted at 2026-08-06 18:40:05 on dist-test-slave-k5rr"
server_key: "62f2a70b5db12040126022d6d5adc68e"
server_key_iv: "76f9b5f3fe6e93ea4583edc819c3b935"
server_key_version: "encryptionkey@0"
I20260806 18:40:05.411742 17067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "e8fd929d5f144d9a9e0f272a09f17295"
format_stamp: "Formatted at 2026-08-06 18:40:05 on dist-test-slave-k5rr"
server_key: "62f2a70b5db12040126022d6d5adc68e"
server_key_iv: "76f9b5f3fe6e93ea4583edc819c3b935"
server_key_version: "encryptionkey@0"
I20260806 18:40:05.415735 17067 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:40:05.418457 17090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:05.419775 17067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:40:05.419898 17067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "e8fd929d5f144d9a9e0f272a09f17295"
format_stamp: "Formatted at 2026-08-06 18:40:05 on dist-test-slave-k5rr"
server_key: "62f2a70b5db12040126022d6d5adc68e"
server_key_iv: "76f9b5f3fe6e93ea4583edc819c3b935"
server_key_version: "encryptionkey@0"
I20260806 18:40:05.420028 17067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:05.433456 17067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:05.434201 17067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:05.434383 17067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:05.441258 17067 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:36341
I20260806 18:40:05.441275 17175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:36341 every 8 connection(s)
I20260806 18:40:05.442238 17067 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:05.445441 17177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:05.445480 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 17067
I20260806 18:40:05.445593 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:05.445856 9159 external_mini_cluster.cc:1428] Setting key 48d88d21779b0a6a384a08fcff87eca4
I20260806 18:40:05.453310 17177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295: Bootstrap starting.
I20260806 18:40:05.456287 17177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:05.456739 17180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:05.447305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.8.241.254:36341)
0806 18:40:05.448080 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:05.448105 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:05.448152 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:40:05.448973 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:05.449036 (+ 63us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:05.449622 (+ 586us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:05.450032 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:05.450524 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.452031 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.452901 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.453624 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.453787 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:05.454608 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:05.454615 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:05.454634 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:05.455157 (+ 523us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:05.455231 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:05.455259 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:05.455356 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:05.455430 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 18:40:05.455729 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":140,"threads_started":1}
I20260806 18:40:05.457267 17177 log.cc:824] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:05.459414 17177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295: No bootstrap required, opened a new log
I20260806 18:40:05.459800 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:40:05.463304 17177 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fd929d5f144d9a9e0f272a09f17295" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36341 } }
I20260806 18:40:05.463560 17177 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:05.463611 17177 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8fd929d5f144d9a9e0f272a09f17295, State: Initialized, Role: FOLLOWER
I20260806 18:40:05.464063 17177 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fd929d5f144d9a9e0f272a09f17295" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36341 } }
I20260806 18:40:05.464197 17177 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:05.464246 17177 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:05.464320 17177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:05.465418 17177 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fd929d5f144d9a9e0f272a09f17295" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36341 } }
I20260806 18:40:05.465798 17177 leader_election.cc:302] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8fd929d5f144d9a9e0f272a09f17295; no voters:
I20260806 18:40:05.466135 17177 leader_election.cc:288] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:05.466238 17186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:05.466456 17186 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [term 1 LEADER]: Becoming Leader. State: Replica: e8fd929d5f144d9a9e0f272a09f17295, State: Running, Role: LEADER
I20260806 18:40:05.466779 17186 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fd929d5f144d9a9e0f272a09f17295" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36341 } }
I20260806 18:40:05.467397 17177 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:05.468430 17190 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8fd929d5f144d9a9e0f272a09f17295. Latest consensus state: current_term: 1 leader_uuid: "e8fd929d5f144d9a9e0f272a09f17295" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fd929d5f144d9a9e0f272a09f17295" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36341 } } }
I20260806 18:40:05.468580 17190 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:05.468845 17188 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8fd929d5f144d9a9e0f272a09f17295" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fd929d5f144d9a9e0f272a09f17295" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36341 } } }
I20260806 18:40:05.468971 17188 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:05.469079 17205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:05.472266 17205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:05.478269 17205 catalog_manager.cc:1347] Generated new cluster ID: c6afc199c33c4caea0584385c4c8495e
I20260806 18:40:05.478348 17205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:05.496582 17205 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:05.497931 17205 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:05.504853 17205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8fd929d5f144d9a9e0f272a09f17295: Generated new TSK 0
I20260806 18:40:05.505688 17205 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:40:05.536801 17236 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 18:40:05.536890 17236 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:40:05.537791 17180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:05.521576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.8.241.254:36341)
0806 18:40:05.521719 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:05.521723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:05.521846 (+ 123us) server_negotiation.cc:407] Connection header received
0806 18:40:05.522021 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:05.522024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:05.522087 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:05.522218 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:40:05.522404 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.523258 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.524165 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.524524 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.524624 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:05.524904 (+ 280us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:40:05.537306 (+ 12402us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44109/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:40:05.537635 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60370: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44109/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 18:40:05.537859 17235 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:05.521447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36341 (local address 127.0.0.1:60370)
0806 18:40:05.521756 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:05.521771 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:05.521878 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:05.522205 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:05.522207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:05.522213 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:40:05.522314 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:05.522319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.523353 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.523356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:05.524058 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:05.524069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.524627 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.524629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:05.524686 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:05.524712 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:40:05.537546 (+ 12834us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44109/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:40:05.537708 (+ 162us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.241.254:36341: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44109/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":113,"threads_started":1}
W20260806 18:40:05.537869 17180 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60370: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44109/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260806 18:40:05.538781 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 17067
2026-08-06T18:40:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (340 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 18:40:05.588425 9159 mini_oidc.cc:152] Starting JWKS server
I20260806 18:40:05.589401 9159 webserver.cc:460] Webserver started at https://127.0.0.1:33387/ using document root <none> and password file <none>
I20260806 18:40:05.589531 9159 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:40:05.589787 9159 webserver.cc:460] Webserver started at http://127.0.0.1:35421/ using document root <none> and password file <none>
2026-08-06T18:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:05Z Disabled control of system clock
I20260806 18:40:05.599810 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:35277
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:35541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:35277
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33387/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:40:05.715507 17244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:05.715837 17244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:05.715881 17244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:05.720039 17244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:05.720124 17244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:05.720137 17244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:05.720152 17244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:05.720166 17244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:40:05.720181 17244 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:40:05.720202 17244 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33387/jwks.json
I20260806 18:40:05.725169 17244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:35541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:35277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:35277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33387/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:05.726344 17244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:05.727897 17244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:05.736639 17260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:05.736639 17255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:05.736888 17244 server_base.cc:1034] running on GCE node
W20260806 18:40:05.736670 17253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:05.737671 17244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:05.738929 17244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:05.740165 17244 hybrid_clock.cc:648] HybridClock initialized: now 1786041605740130 us; error 47 us; skew 500 ppm
I20260806 18:40:05.742221 17244 webserver.cc:460] Webserver started at http://127.8.241.254:37631/ using document root <none> and password file <none>
I20260806 18:40:05.742821 17244 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:05.742897 17244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:05.743113 17244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:05.745318 17244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "1b3f124ec1de462fbcc766e92156305a"
format_stamp: "Formatted at 2026-08-06 18:40:05 on dist-test-slave-k5rr"
server_key: "e934668f7451fb21d9437e83bdb10250"
server_key_iv: "55ac125266221ac07deb58dc3a47f17a"
server_key_version: "encryptionkey@0"
I20260806 18:40:05.745891 17244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "1b3f124ec1de462fbcc766e92156305a"
format_stamp: "Formatted at 2026-08-06 18:40:05 on dist-test-slave-k5rr"
server_key: "e934668f7451fb21d9437e83bdb10250"
server_key_iv: "55ac125266221ac07deb58dc3a47f17a"
server_key_version: "encryptionkey@0"
I20260806 18:40:05.749966 17244 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:40:05.752575 17271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:05.753882 17244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:05.754009 17244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "1b3f124ec1de462fbcc766e92156305a"
format_stamp: "Formatted at 2026-08-06 18:40:05 on dist-test-slave-k5rr"
server_key: "e934668f7451fb21d9437e83bdb10250"
server_key_iv: "55ac125266221ac07deb58dc3a47f17a"
server_key_version: "encryptionkey@0"
I20260806 18:40:05.754117 17244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:05.766770 17244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:05.767535 17244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:05.767755 17244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:05.775274 17244 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:35277
I20260806 18:40:05.775280 17357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:35277 every 8 connection(s)
I20260806 18:40:05.776400 17244 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:05.779507 17359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:05.785393 17359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a: Bootstrap starting.
I20260806 18:40:05.785898 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 17244
I20260806 18:40:05.786003 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:05.786267 9159 external_mini_cluster.cc:1428] Setting key c31e4ca55e7bd10bf36954a9979b287a
I20260806 18:40:05.788178 17359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:05.788956 17359 log.cc:824] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:05.790944 17359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a: No bootstrap required, opened a new log
I20260806 18:40:05.795224 17359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3f124ec1de462fbcc766e92156305a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35277 } }
I20260806 18:40:05.795483 17359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:05.795543 17359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b3f124ec1de462fbcc766e92156305a, State: Initialized, Role: FOLLOWER
I20260806 18:40:05.795990 17359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3f124ec1de462fbcc766e92156305a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35277 } }
I20260806 18:40:05.796128 17359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:05.796195 17359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:05.796241 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:05.787575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36640 (local address 127.8.241.254:35277)
0806 18:40:05.788282 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:05.788302 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:05.788348 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:40:05.789064 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:05.789090 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:05.789585 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:05.789905 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:05.790381 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.791403 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.792314 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.793095 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.793275 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:05.794239 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:05.794245 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:05.794260 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:05.794672 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:05.794731 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:05.794753 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:05.794830 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:05.795020 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 18:40:05.795308 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":134,"threads_started":1}
I20260806 18:40:05.796293 17359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:05.797505 17359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3f124ec1de462fbcc766e92156305a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35277 } }
I20260806 18:40:05.797855 17359 leader_election.cc:302] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b3f124ec1de462fbcc766e92156305a; no voters:
I20260806 18:40:05.798207 17359 leader_election.cc:288] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:05.798305 17368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:05.798565 17368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [term 1 LEADER]: Becoming Leader. State: Replica: 1b3f124ec1de462fbcc766e92156305a, State: Running, Role: LEADER
I20260806 18:40:05.798942 17368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3f124ec1de462fbcc766e92156305a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35277 } }
I20260806 18:40:05.799423 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:40:05.799664 17359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:05.800206 17369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b3f124ec1de462fbcc766e92156305a. Latest consensus state: current_term: 1 leader_uuid: "1b3f124ec1de462fbcc766e92156305a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3f124ec1de462fbcc766e92156305a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35277 } } }
I20260806 18:40:05.800349 17369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:05.800701 17372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b3f124ec1de462fbcc766e92156305a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3f124ec1de462fbcc766e92156305a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35277 } } }
I20260806 18:40:05.800799 17372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:05.801250 17377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:05.804610 17377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:05.810810 17377 catalog_manager.cc:1347] Generated new cluster ID: 64c49568b18b48d090f967fb853789ca
I20260806 18:40:05.810900 17377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:05.822975 17377 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:05.823952 17377 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:05.836481 17377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b3f124ec1de462fbcc766e92156305a: Generated new TSK 0
I20260806 18:40:05.837417 17377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:40:05.858922 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:05.853649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36648 (local address 127.8.241.254:35277)
0806 18:40:05.853794 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:05.853799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:05.854025 (+ 226us) server_negotiation.cc:407] Connection header received
0806 18:40:05.854264 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:05.854266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:05.854318 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:05.854414 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:40:05.854664 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.855187 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.856404 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.856625 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.856698 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:05.857049 (+ 351us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:40:05.858514 (+ 1465us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:40:05.858757 (+ 243us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 18:40:05.859087 17366 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 18:40:05.860812 17415 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:05.853556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:35277 (local address 127.0.0.1:36648)
0806 18:40:05.853901 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:05.853924 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:05.854118 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:05.854432 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:05.854436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:05.854444 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:40:05.854575 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:05.854582 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.855285 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.855289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:05.856295 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:05.856306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:05.856731 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:05.856735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:05.856824 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:05.856856 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:40:05.858702 (+ 1846us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:40:05.860624 (+ 1922us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.241.254:35277: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":1702,"thread_start_us":104,"threads_started":1}
I20260806 18:40:05.861402 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 17244
E20260806 18:40:05.877123 17417 webserver.cc:533] Webserver: sslize failed:
E20260806 18:40:05.877215 17417 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T18:40:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (321 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T18:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:05Z Disabled control of system clock
I20260806 18:40:05.919124 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39679
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:36559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39679
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/testchainca.cert with env {}
W20260806 18:40:06.035972 17423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:06.036257 17423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:06.036296 17423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:06.040400 17423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:06.040488 17423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:06.040508 17423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:06.040524 17423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:06.040537 17423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:40:06.040558 17423 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:40:06.040575 17423 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 18:40:06.046200 17423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39679
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:06.047331 17423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:06.048398 17423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:06.057511 17433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:06.057487 17430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:06.057484 17429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:06.057747 17423 server_base.cc:1034] running on GCE node
I20260806 18:40:06.058514 17423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:06.059748 17423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:06.060902 17423 hybrid_clock.cc:648] HybridClock initialized: now 1786041606060872 us; error 46 us; skew 500 ppm
I20260806 18:40:06.063006 17423 webserver.cc:460] Webserver started at http://127.8.241.254:33657/ using document root <none> and password file <none>
I20260806 18:40:06.063632 17423 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:06.063704 17423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:06.063923 17423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:06.066216 17423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "f681fdd9978d4ce69a1e0457f8d10601"
format_stamp: "Formatted at 2026-08-06 18:40:06 on dist-test-slave-k5rr"
server_key: "e6935acca19364cc0c09749966f4106d"
server_key_iv: "373a481e3382f1a04296e8a62759f93a"
server_key_version: "encryptionkey@0"
I20260806 18:40:06.066789 17423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "f681fdd9978d4ce69a1e0457f8d10601"
format_stamp: "Formatted at 2026-08-06 18:40:06 on dist-test-slave-k5rr"
server_key: "e6935acca19364cc0c09749966f4106d"
server_key_iv: "373a481e3382f1a04296e8a62759f93a"
server_key_version: "encryptionkey@0"
I20260806 18:40:06.070677 17423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:40:06.073280 17444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:06.074604 17423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:40:06.074750 17423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "f681fdd9978d4ce69a1e0457f8d10601"
format_stamp: "Formatted at 2026-08-06 18:40:06 on dist-test-slave-k5rr"
server_key: "e6935acca19364cc0c09749966f4106d"
server_key_iv: "373a481e3382f1a04296e8a62759f93a"
server_key_version: "encryptionkey@0"
I20260806 18:40:06.074867 17423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:06.101841 17423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:06.102622 17423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:06.102823 17423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:06.110038 17423 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:39679
I20260806 18:40:06.110047 17529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:39679 every 8 connection(s)
I20260806 18:40:06.111096 17423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:06.114557 17530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:06.115881 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 17423
I20260806 18:40:06.116024 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:06.116350 9159 external_mini_cluster.cc:1428] Setting key ccb970e68bb94ee626235eb34cde3a47
I20260806 18:40:06.121219 17530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601: Bootstrap starting.
I20260806 18:40:06.123605 17530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:06.124516 17530 log.cc:824] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:06.126015 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:06.117912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58310 (local address 127.8.241.254:39679)
0806 18:40:06.118632 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:06.118648 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:06.118686 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:40:06.119342 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:06.119361 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:06.119779 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:06.120107 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:06.120558 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:06.121629 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:06.122498 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:06.122981 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:06.123129 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:06.124027 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:06.124034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:06.124050 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:06.124554 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:06.124633 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:06.124663 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:06.124768 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:06.124873 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:40:06.125204 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":144,"threads_started":1}
I20260806 18:40:06.126703 17530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601: No bootstrap required, opened a new log
I20260806 18:40:06.128775 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:40:06.130712 17530 raft_consensus.cc:348] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fdd9978d4ce69a1e0457f8d10601" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39679 } }
I20260806 18:40:06.130963 17530 raft_consensus.cc:374] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:06.131034 17530 raft_consensus.cc:729] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f681fdd9978d4ce69a1e0457f8d10601, State: Initialized, Role: FOLLOWER
I20260806 18:40:06.131470 17530 consensus_queue.cc:260] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fdd9978d4ce69a1e0457f8d10601" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39679 } }
I20260806 18:40:06.131609 17530 raft_consensus.cc:388] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:06.131678 17530 raft_consensus.cc:482] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:06.131781 17530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:06.132746 17530 raft_consensus.cc:504] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fdd9978d4ce69a1e0457f8d10601" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39679 } }
I20260806 18:40:06.133157 17530 leader_election.cc:302] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f681fdd9978d4ce69a1e0457f8d10601; no voters:
I20260806 18:40:06.133517 17530 leader_election.cc:288] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:06.133601 17538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:06.133824 17538 raft_consensus.cc:686] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [term 1 LEADER]: Becoming Leader. State: Replica: f681fdd9978d4ce69a1e0457f8d10601, State: Running, Role: LEADER
I20260806 18:40:06.134133 17538 consensus_queue.cc:237] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fdd9978d4ce69a1e0457f8d10601" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39679 } }
I20260806 18:40:06.134786 17530 sys_catalog.cc:564] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:06.135797 17539 sys_catalog.cc:455] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f681fdd9978d4ce69a1e0457f8d10601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fdd9978d4ce69a1e0457f8d10601" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39679 } } }
I20260806 18:40:06.135931 17539 sys_catalog.cc:458] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:06.136309 17561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:06.139721 17561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:06.136263 17540 sys_catalog.cc:455] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f681fdd9978d4ce69a1e0457f8d10601. Latest consensus state: current_term: 1 leader_uuid: "f681fdd9978d4ce69a1e0457f8d10601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fdd9978d4ce69a1e0457f8d10601" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39679 } } }
I20260806 18:40:06.141088 17540 sys_catalog.cc:458] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:06.146049 17561 catalog_manager.cc:1347] Generated new cluster ID: 7f6f809c5b5542cea86c8b06aaa43a84
I20260806 18:40:06.146134 17561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:06.191773 17561 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:06.192821 17561 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:06.206615 17561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f681fdd9978d4ce69a1e0457f8d10601: Generated new TSK 0
I20260806 18:40:06.207634 17561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:40:06.299366 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:06.225635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.8.241.254:39679)
0806 18:40:06.225783 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:06.225788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:06.225904 (+ 116us) server_negotiation.cc:407] Connection header received
0806 18:40:06.226075 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:06.226079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:06.226147 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:06.226289 (+ 142us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:40:06.226538 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:06.227347 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:06.228236 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:06.228595 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:06.228697 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:06.229101 (+ 404us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:40:06.298929 (+ 69828us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:40:06.299233 (+ 304us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58320: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 18:40:06.299465 17533 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58320: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 18:40:06.299611 17586 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:06.225490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39679 (local address 127.0.0.1:58320)
0806 18:40:06.225809 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:06.225823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:06.225930 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:06.226302 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:06.226305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:06.226313 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:40:06.226427 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:06.226433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:06.227488 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:06.227491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:06.228117 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:06.228125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:06.228764 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:06.228768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:06.228850 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:06.228879 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:40:06.299215 (+ 70336us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:40:06.299404 (+ 189us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.241.254:39679: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":111,"threads_started":1}
I20260806 18:40:06.300173 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 17423
2026-08-06T18:40:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (419 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (167 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (134 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:40:06 dist-test-slave-k5rr krb5kdc[17601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:40:06 dist-test-slave-k5rr krb5kdc[17601](info): set up 2 sockets
Aug 06 18:40:06 dist-test-slave-k5rr krb5kdc[17601](info): commencing operation
krb5kdc: starting...
W20260806 18:40:07.753551 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:40:07 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:40:07.876390 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37817
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:37817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:07.990097 17623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:07.990370 17623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:07.990427 17623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:07.994462 17623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:07.994519 17623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:07.994546 17623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:07.994566 17623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:07.994587 17623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:07.999758 17623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:37817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:08.000804 17623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:08.001848 17623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:08.010241 17632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:08.010309 17623 server_base.cc:1034] running on GCE node
W20260806 18:40:08.010298 17634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:08.010252 17631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:08.011129 17623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:08.012280 17623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:08.013439 17623 hybrid_clock.cc:648] HybridClock initialized: now 1786041608013425 us; error 32 us; skew 500 ppm
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.016157 17623 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:40:08.017071 17623 webserver.cc:460] Webserver started at http://127.8.241.254:33149/ using document root <none> and password file <none>
I20260806 18:40:08.017607 17623 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:08.017666 17623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:08.017832 17623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:08.019858 17623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "5f8ac9d6be0f4df39c0525fedd157424"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "9c1e10cdf0a774c0780140bbd3212beb"
server_key_iv: "db23d23fdf9b76afb6a7b8936a00906c"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.020373 17623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "5f8ac9d6be0f4df39c0525fedd157424"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "9c1e10cdf0a774c0780140bbd3212beb"
server_key_iv: "db23d23fdf9b76afb6a7b8936a00906c"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.024039 17623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:40:08.026403 17643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.027601 17623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:40:08.027730 17623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "5f8ac9d6be0f4df39c0525fedd157424"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "9c1e10cdf0a774c0780140bbd3212beb"
server_key_iv: "db23d23fdf9b76afb6a7b8936a00906c"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.027868 17623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:08.045326 17623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:08.048121 17623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:08.048295 17623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:08.055160 17623 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:37817
I20260806 18:40:08.055191 17738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:37817 every 8 connection(s)
I20260806 18:40:08.056087 17623 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:08.058892 17739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:08.062767 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 17623
I20260806 18:40:08.062868 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:08.063126 9159 external_mini_cluster.cc:1428] Setting key b6343ae7da8d5eea522b6a91f90b01c1
I20260806 18:40:08.064823 17739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424: Bootstrap starting.
I20260806 18:40:08.067603 17739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:08.068511 17739 log.cc:824] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:08.070410 17739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424: No bootstrap required, opened a new log
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:08.074194 17739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8ac9d6be0f4df39c0525fedd157424" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37817 } }
I20260806 18:40:08.074455 17739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:08.074532 17739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f8ac9d6be0f4df39c0525fedd157424, State: Initialized, Role: FOLLOWER
I20260806 18:40:08.075008 17739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8ac9d6be0f4df39c0525fedd157424" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37817 } }
I20260806 18:40:08.075150 17739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:08.075217 17739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:08.075315 17739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:08.076174 17739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8ac9d6be0f4df39c0525fedd157424" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37817 } }
I20260806 18:40:08.076519 17739 leader_election.cc:302] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f8ac9d6be0f4df39c0525fedd157424; no voters:
I20260806 18:40:08.076822 17739 leader_election.cc:288] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:08.076922 17745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:08.077167 17745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [term 1 LEADER]: Becoming Leader. State: Replica: 5f8ac9d6be0f4df39c0525fedd157424, State: Running, Role: LEADER
I20260806 18:40:08.077546 17745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8ac9d6be0f4df39c0525fedd157424" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37817 } }
I20260806 18:40:08.077661 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.064534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56606 (local address 127.8.241.254:37817)
0806 18:40:08.064962 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.064982 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.065045 (+ 63us) server_negotiation.cc:407] Connection header received
0806 18:40:08.065927 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.065952 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.066389 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.066752 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.067448 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.068924 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.069789 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.070248 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.070394 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.072713 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:08.072726 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:08.072739 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:08.072766 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:08.074469 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.074948 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.074953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.074957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.075068 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.075333 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.075337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.075338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.075749 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:08.075952 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:08.076241 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 18:40:08.076562 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
I20260806 18:40:08.078147 17739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:08.078786 17745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f8ac9d6be0f4df39c0525fedd157424. Latest consensus state: current_term: 1 leader_uuid: "5f8ac9d6be0f4df39c0525fedd157424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8ac9d6be0f4df39c0525fedd157424" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37817 } } }
I20260806 18:40:08.078970 17745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:08.079205 17747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f8ac9d6be0f4df39c0525fedd157424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f8ac9d6be0f4df39c0525fedd157424" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37817 } } }
I20260806 18:40:08.079306 17747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:08.079900 17757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:08.083024 17757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:08.088773 17757 catalog_manager.cc:1347] Generated new cluster ID: acf7ce31c7ab45d495267e10b5ce3a1d
I20260806 18:40:08.088832 17757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:08.105424 17757 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:08.106405 17757 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:08.113339 17757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f8ac9d6be0f4df39c0525fedd157424: Generated new TSK 0
I20260806 18:40:08.114059 17757 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:40:08.152442 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37817
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:08.288952 17777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:08.289230 17777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:08.289274 17777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:08.293210 17777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:08.293264 17777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:08.293320 17777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:08.298399 17777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:08.299386 17777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:08.300606 17777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:08.309968 17785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:08.310004 17783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:08.310195 17777 server_base.cc:1034] running on GCE node
W20260806 18:40:08.310305 17787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:08.310671 17777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:08.311250 17777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:08.312413 17777 hybrid_clock.cc:648] HybridClock initialized: now 1786041608312392 us; error 31 us; skew 500 ppm
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.315052 17777 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:40:08.315907 17777 webserver.cc:460] Webserver started at http://127.8.241.193:37235/ using document root <none> and password file <none>
I20260806 18:40:08.316469 17777 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:08.316537 17777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:08.316731 17777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:08.318796 17777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "820561d5b84d4cf39e02119ffa4353af"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "64975ce026201b81df066db0c2bc0e9e"
server_key_iv: "fa1c31a757f253d64475569a461fe4f3"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.319346 17777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "820561d5b84d4cf39e02119ffa4353af"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "64975ce026201b81df066db0c2bc0e9e"
server_key_iv: "fa1c31a757f253d64475569a461fe4f3"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.323132 17777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:40:08.325670 17796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.326957 17777 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:08.327071 17777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "820561d5b84d4cf39e02119ffa4353af"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "64975ce026201b81df066db0c2bc0e9e"
server_key_iv: "fa1c31a757f253d64475569a461fe4f3"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.327173 17777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:08.353626 17777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:08.356531 17777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:08.356742 17777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:08.357434 17777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:08.358333 17777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:08.358387 17777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.358448 17777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:08.358476 17777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.367659 17777 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:39171
I20260806 18:40:08.367686 17979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:39171 every 8 connection(s)
I20260806 18:40:08.368585 17777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:08.370462 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 17777
I20260806 18:40:08.370589 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:08.370919 9159 external_mini_cluster.cc:1428] Setting key 4ebd76ca0c0a31abf52c479ae89624b4
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:40:08.382819 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.370443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:60353 (local address 127.8.241.254:37817)
0806 18:40:08.370590 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.370595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.371650 (+ 1055us) server_negotiation.cc:407] Connection header received
0806 18:40:08.373325 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.373328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.373383 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.373458 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.374672 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.375201 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.376327 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.376554 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.376641 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.379261 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:08.379264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:08.379266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:08.379291 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:08.380701 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.381213 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.381215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.381217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.381294 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.381701 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.381704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.381706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.381900 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:08.381988 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:08.382475 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 18:40:08.382580 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:40:08.383621 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.370907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37817 (local address 127.8.241.193:60353)
0806 18:40:08.371490 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:08.371552 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:08.373112 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:08.373657 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:08.373665 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:08.374026 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:08.374520 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:08.374535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.375325 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.375328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:08.376206 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:08.376216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.376697 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.376701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:08.376860 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.377426 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:08.377443 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:08.379068 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:08.380810 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:08.380817 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:08.380832 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:08.381135 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:08.381403 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:08.381406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:08.381408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:08.381593 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:08.382022 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:08.382028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:08.382275 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 18:40:08.382428 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 18:40:08.382684 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":145,"threads_started":1}
I20260806 18:40:08.384877 17980 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37817
I20260806 18:40:08.385150 17980 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:08.385643 17980 heartbeater.cc:511] Master 127.8.241.254:37817 requested a full tablet report, sending...
I20260806 18:40:08.387413 17675 ts_manager.cc:194] Registered new tserver with Master: 820561d5b84d4cf39e02119ffa4353af (127.8.241.193:39171)
I20260806 18:40:08.388994 17675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:60353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:40:08.399748 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37817
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:08.511061 17988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:08.511318 17988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:08.511360 17988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:08.515309 17988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:08.515358 17988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:08.515414 17988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:08.520556 17988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:08.521590 17988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:08.522559 17988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:08.532333 18000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:08.532400 17988 server_base.cc:1034] running on GCE node
W20260806 18:40:08.532335 17998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:08.532384 17997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:08.532981 17988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:08.533623 17988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:08.534783 17988 hybrid_clock.cc:648] HybridClock initialized: now 1786041608534763 us; error 31 us; skew 500 ppm
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.537492 17988 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:40:08.538386 17988 webserver.cc:460] Webserver started at http://127.8.241.194:36629/ using document root <none> and password file <none>
I20260806 18:40:08.538956 17988 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:08.539028 17988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:08.539242 17988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:08.541414 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "4150789db42648c5a2da65c3196cc23b"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "9049daa1b22d7e7fd5c466e8d92b68f2"
server_key_iv: "5486f8536cddb45823ffb0da4f749a25"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.542008 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "4150789db42648c5a2da65c3196cc23b"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "9049daa1b22d7e7fd5c466e8d92b68f2"
server_key_iv: "5486f8536cddb45823ffb0da4f749a25"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.545873 17988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:40:08.548343 18009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.549605 17988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:40:08.549714 17988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "4150789db42648c5a2da65c3196cc23b"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "9049daa1b22d7e7fd5c466e8d92b68f2"
server_key_iv: "5486f8536cddb45823ffb0da4f749a25"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.549803 17988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:08.572659 17988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:08.575549 17988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:08.575714 17988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:08.576293 17988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:08.577169 17988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:08.577224 17988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.577265 17988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:08.577286 17988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.585917 17988 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:38131
I20260806 18:40:08.585947 18188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:38131 every 8 connection(s)
I20260806 18:40:08.586805 17988 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:08.595878 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 17988
I20260806 18:40:08.595980 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:08.596251 9159 external_mini_cluster.cc:1428] Setting key ba63f08b98075455ffee4cc2f30142d8
I20260806 18:40:08.599992 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.588571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:48005 (local address 127.8.241.254:37817)
0806 18:40:08.588687 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.588690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.589582 (+ 892us) server_negotiation.cc:407] Connection header received
0806 18:40:08.590983 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.590987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.591045 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.591118 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.592339 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.592865 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.594033 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.594256 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.594347 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.596677 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:08.596679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:08.596681 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:08.596705 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:08.598049 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.598527 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.598531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.598532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.598607 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.598923 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.598926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.598928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.599110 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:08.599198 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:08.599702 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 18:40:08.599828 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:40:08.600847 18194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.588935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37817 (local address 127.8.241.194:48005)
0806 18:40:08.589425 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:08.589476 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:08.590761 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:08.591305 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:08.591313 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:08.591695 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:08.592186 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:08.592200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.592989 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.592992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:08.593906 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:08.593917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.594399 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.594404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:08.594582 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.595073 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:08.595089 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:08.596519 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:08.598158 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:08.598165 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:08.598180 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:08.598430 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:08.598691 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:08.598694 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:08.598696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:08.598835 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:08.599212 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:08.599219 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:08.599467 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 18:40:08.599636 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 18:40:08.599976 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":118,"threads_started":1}
I20260806 18:40:08.602005 18190 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37817
I20260806 18:40:08.602231 18190 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:08.602706 18190 heartbeater.cc:511] Master 127.8.241.254:37817 requested a full tablet report, sending...
I20260806 18:40:08.603731 17675 ts_manager.cc:194] Registered new tserver with Master: 4150789db42648c5a2da65c3196cc23b (127.8.241.194:38131)
I20260806 18:40:08.604292 17675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:48005
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:40:08.623973 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37817
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:08.738826 18197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:08.739106 18197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:08.739167 18197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:08.743255 18197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:08.743327 18197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:08.743409 18197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:08.748708 18197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:08.749806 18197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:08.750885 18197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:08.760815 18210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:08.760784 18206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:08.760986 18197 server_base.cc:1034] running on GCE node
W20260806 18:40:08.760795 18207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:08.761562 18197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:08.762202 18197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:08.763381 18197 hybrid_clock.cc:648] HybridClock initialized: now 1786041608763354 us; error 36 us; skew 500 ppm
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.766079 18197 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:40:08.766917 18197 webserver.cc:460] Webserver started at http://127.8.241.195:44657/ using document root <none> and password file <none>
I20260806 18:40:08.767482 18197 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:08.767549 18197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:08.767736 18197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:08.769955 18197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "f89c9226c093464ca249d2fd745a2a03"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "bc4b3004b08f54264d567f103d21e916"
server_key_iv: "ea64da0675a581bcabaab6e9f9c958c6"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.770491 18197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "f89c9226c093464ca249d2fd745a2a03"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "bc4b3004b08f54264d567f103d21e916"
server_key_iv: "ea64da0675a581bcabaab6e9f9c958c6"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.774149 18197 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 18:40:08.776432 18222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.777639 18197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:08.777760 18197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "f89c9226c093464ca249d2fd745a2a03"
format_stamp: "Formatted at 2026-08-06 18:40:08 on dist-test-slave-k5rr"
server_key: "bc4b3004b08f54264d567f103d21e916"
server_key_iv: "ea64da0675a581bcabaab6e9f9c958c6"
server_key_version: "encryptionkey@0"
I20260806 18:40:08.777853 18197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:08.788560 18197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:08.791718 18197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:08.791898 18197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:08.792510 18197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:08.793377 18197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:08.793424 18197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.793478 18197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:08.793509 18197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:08.801720 18197 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:37153
I20260806 18:40:08.801766 18402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:37153 every 8 connection(s)
I20260806 18:40:08.802618 18197 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:08.810488 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 18197
I20260806 18:40:08.810600 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:08.810899 9159 external_mini_cluster.cc:1428] Setting key 96611a2e9aa57e0c677c553a170bc33c
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:08.815932 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.804416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:49547 (local address 127.8.241.254:37817)
0806 18:40:08.804567 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.804571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.805423 (+ 852us) server_negotiation.cc:407] Connection header received
0806 18:40:08.806811 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.806814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.806874 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.806946 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.808253 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.808784 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.809961 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.810178 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.810267 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.812764 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:08.812767 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:08.812770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:08.812798 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:08.813996 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.814464 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.814468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.814470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.814552 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.814862 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.814865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.814867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.815063 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:08.815149 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:08.815653 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 18:40:08.815775 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:40:08.816730 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.804786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37817 (local address 127.8.241.195:49547)
0806 18:40:08.805275 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:08.805320 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:08.806598 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:08.807141 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:08.807149 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:08.807593 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:08.808109 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:08.808125 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.808906 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.808909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:08.809841 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:08.809852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.810338 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.810343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:08.810503 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.811119 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:08.811136 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:08.812602 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:08.814102 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:08.814108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:08.814125 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:08.814371 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:08.814631 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:08.814634 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:08.814636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:08.814775 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:08.815159 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:08.815167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:08.815413 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 18:40:08.815594 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:40:08.815916 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":157,"threads_started":1}
I20260806 18:40:08.817821 18405 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37817
I20260806 18:40:08.818086 18405 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:08.818593 18405 heartbeater.cc:511] Master 127.8.241.254:37817 requested a full tablet report, sending...
I20260806 18:40:08.819649 17675 ts_manager.cc:194] Registered new tserver with Master: f89c9226c093464ca249d2fd745a2a03 (127.8.241.195:37153)
I20260806 18:40:08.820188 17675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:49547
I20260806 18:40:08.824677 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/test-admin.keytab.
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.840641 9159 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 18:40:08.843328 18419 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:08.841994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60382 (local address 127.8.241.193:39171)
0806 18:40:08.842293 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.842313 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.842345 (+ 32us) server_negotiation.cc:407] Connection header received
0806 18:40:08.842404 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.842409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.842631 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.842753 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.843142 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60382: BlockingRecv error: recv got EOF from 127.0.0.1:60382 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
W20260806 18:40:08.843609 9159 init.cc:244] Error while opening credential cache '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5cc)
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.844568 9159 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.845434 9159 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:40:08.854236 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.846274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60390 (local address 127.8.241.193:39171)
0806 18:40:08.846383 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.846387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.846443 (+ 56us) server_negotiation.cc:407] Connection header received
0806 18:40:08.846562 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.846566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.846607 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.846670 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.847205 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.847972 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.848786 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.849188 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.849283 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.851298 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:08.851307 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:08.851320 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:08.851343 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:08.852642 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.853074 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.853079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.853081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.853158 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.853361 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.853363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.853364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.853599 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:08.853738 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:08.853982 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 18:40:08.854111 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:40:08.857748 18419 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:08.856858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60400 (local address 127.8.241.193:39171)
0806 18:40:08.856966 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.856970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.857055 (+ 85us) server_negotiation.cc:407] Connection header received
0806 18:40:08.857180 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.857182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.857215 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.857275 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.857659 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60400: BlockingRecv error: recv got EOF from 127.0.0.1:60400 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 18:40:08.857841 9159 init.cc:244] Error while opening credential cache '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5cc)
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.858714 9159 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.859479 9159 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:40:08.866963 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.860197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60412 (local address 127.8.241.193:39171)
0806 18:40:08.860307 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.860310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.860377 (+ 67us) server_negotiation.cc:407] Connection header received
0806 18:40:08.860478 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.860480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.860514 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.860566 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.860950 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.861483 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.862221 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.862430 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.862504 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.864394 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:08.864397 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:08.864399 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:08.864420 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:08.865719 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.866053 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.866056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.866057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.866129 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.866305 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.866306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.866308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.866457 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:08.866573 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:08.866724 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 18:40:08.866817 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 18:40:08.869380 18419 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:08.868446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.8.241.193:39171)
0806 18:40:08.868559 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.868563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.868653 (+ 90us) server_negotiation.cc:407] Connection header received
0806 18:40:08.868767 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.868770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.868808 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.868867 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.869289 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60414: BlockingRecv error: recv got EOF from 127.0.0.1:60414 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 18:40:08.869486 9159 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041577757111-9159-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.870327 9159 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:08.871052 9159 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[17601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:40:08.878343 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:08.871749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.8.241.193:39171)
0806 18:40:08.871859 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:08.871862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:08.871949 (+ 87us) server_negotiation.cc:407] Connection header received
0806 18:40:08.872028 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:08.872030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:08.872061 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:08.872119 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:08.872514 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.872977 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.873684 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:08.873879 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:08.873960 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:08.875872 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:08.875875 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:08.875877 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:08.875897 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:08.876919 (+ 1022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.877322 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.877325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.877327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.877396 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:08.877635 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:08.877639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:08.877640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:08.877789 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:08.877896 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:08.878114 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 18:40:08.878212 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:40:08.879297 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 17777
I20260806 18:40:08.897898 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 17988
I20260806 18:40:08.906103 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 18197
I20260806 18:40:08.917878 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 17623
2026-08-06T18:40:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2311 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[18438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[18438](info): set up 2 sockets
Aug 06 18:40:08 dist-test-slave-k5rr krb5kdc[18438](info): commencing operation
krb5kdc: starting...
W20260806 18:40:10.050455 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:40:10 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.254.keytab.
Entry for principal oryx/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.254.keytab.
I20260806 18:40:10.171052 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:43515
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.254.keytab
--principal=oryx/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:43515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:10.282148 18459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:10.282426 18459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:10.282482 18459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:10.286648 18459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:10.286721 18459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:10.286753 18459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:10.286773 18459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:10.286794 18459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:10.292068 18459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:43515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.254.keytab
--principal=oryx/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:43515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:10.293157 18459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:10.294194 18459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:10.302369 18465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:10.302345 18471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:10.302366 18467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:10.302546 18459 server_base.cc:1034] running on GCE node
I20260806 18:40:10.303134 18459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:10.304168 18459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:10.305315 18459 hybrid_clock.cc:648] HybridClock initialized: now 1786041610305290 us; error 38 us; skew 500 ppm
Aug 06 18:40:10 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:10.307834 18459 init.cc:377] Logged in from keytab as oryx/127.8.241.254@KRBTEST.COM (short username oryx)
I20260806 18:40:10.308660 18459 webserver.cc:460] Webserver started at http://127.8.241.254:39329/ using document root <none> and password file <none>
I20260806 18:40:10.309223 18459 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:10.309291 18459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:10.309496 18459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:10.311497 18459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "7bc7173999834375bfaeaf8f77292c24"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "61f5e0b8dcbcb598fcb7738376bc6728"
server_key_iv: "c40257e5822dd3fc5a7e14b71eb7e59a"
server_key_version: "encryptionkey@0"
I20260806 18:40:10.312038 18459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "7bc7173999834375bfaeaf8f77292c24"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "61f5e0b8dcbcb598fcb7738376bc6728"
server_key_iv: "c40257e5822dd3fc5a7e14b71eb7e59a"
server_key_version: "encryptionkey@0"
I20260806 18:40:10.315577 18459 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:40:10.317901 18481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:10.319067 18459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:10.319191 18459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "7bc7173999834375bfaeaf8f77292c24"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "61f5e0b8dcbcb598fcb7738376bc6728"
server_key_iv: "c40257e5822dd3fc5a7e14b71eb7e59a"
server_key_version: "encryptionkey@0"
I20260806 18:40:10.319283 18459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:10.337693 18459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:10.340299 18459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:10.340476 18459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:10.347163 18459 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:43515
I20260806 18:40:10.347421 18577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:43515 every 8 connection(s)
I20260806 18:40:10.348119 18459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:10.350895 18578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:10.356369 18578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: Bootstrap starting.
I20260806 18:40:10.356968 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 18459
I20260806 18:40:10.357100 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:10.357384 9159 external_mini_cluster.cc:1428] Setting key 4bdfca92f6969fb2d69d59a95c964d02
I20260806 18:40:10.359110 18578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:10.359989 18578 log.cc:824] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:10.361917 18578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: No bootstrap required, opened a new log
I20260806 18:40:10.365351 18578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } }
I20260806 18:40:10.365578 18578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:10.365650 18578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc7173999834375bfaeaf8f77292c24, State: Initialized, Role: FOLLOWER
Aug 06 18:40:10 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:10.366101 18578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } }
I20260806 18:40:10.366274 18578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:10.366345 18578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:10.366458 18578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:10.367466 18578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } }
I20260806 18:40:10.367815 18578 leader_election.cc:302] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bc7173999834375bfaeaf8f77292c24; no voters:
I20260806 18:40:10.368086 18578 leader_election.cc:288] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:10.368221 18588 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:10.368441 18588 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 1 LEADER]: Becoming Leader. State: Replica: 7bc7173999834375bfaeaf8f77292c24, State: Running, Role: LEADER
I20260806 18:40:10.368788 18588 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } }
I20260806 18:40:10.369374 18578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:10.370427 18590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bc7173999834375bfaeaf8f77292c24. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } }
I20260806 18:40:10.370555 18590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:10.370855 18589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } }
I20260806 18:40:10.370935 18589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:10.371337 18600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:10.371765 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:10.358650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57640 (local address 127.8.241.254:43515)
0806 18:40:10.359063 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:10.359083 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:10.359128 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:40:10.359971 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:10.359997 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:10.360422 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:10.360742 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:10.361581 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.362597 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.363429 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.363901 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.364059 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:10.366590 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:10.366597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:10.366610 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:10.366638 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:10.368277 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:10.368754 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:10.368760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:10.368766 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:10.368916 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:10.369254 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:10.369258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:10.369259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:10.369773 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:10.369966 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:10.370437 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 18:40:10.370747 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":104,"threads_started":1}
I20260806 18:40:10.374348 18600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:10.379510 18600 catalog_manager.cc:1347] Generated new cluster ID: e2853932c97c4ac2b942f359e8756eb2
I20260806 18:40:10.379581 18600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:10.402603 18600 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:10.403584 18600 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:10.409359 18600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: Generated new TSK 0
I20260806 18:40:10.410048 18600 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab.
Entry for principal oryx/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab.
I20260806 18:40:10.444252 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43515
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab
--principal=oryx/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:10.582943 18629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:10.583191 18629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:10.583237 18629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:10.587170 18629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:10.587230 18629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:10.587296 18629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:10.592497 18629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab
--principal=oryx/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:10.593518 18629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:10.594544 18629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:10.604038 18642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:10.604012 18641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:10.604018 18647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:10.604314 18629 server_base.cc:1034] running on GCE node
I20260806 18:40:10.604710 18629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:10.605397 18629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:10.606571 18629 hybrid_clock.cc:648] HybridClock initialized: now 1786041610606541 us; error 39 us; skew 500 ppm
Aug 06 18:40:10 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:10.613840 18629 init.cc:377] Logged in from keytab as oryx/127.8.241.193@KRBTEST.COM (short username oryx)
I20260806 18:40:10.614770 18629 webserver.cc:460] Webserver started at http://127.8.241.193:35227/ using document root <none> and password file <none>
I20260806 18:40:10.615350 18629 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:10.615419 18629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:10.615648 18629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:10.617844 18629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "c6ff04d70057445c801791a432a44a1f"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "b67951fef0b68c9acc7f5fb6ae95f5ea"
server_key_iv: "2ec924b092e13246499a72eacd678cca"
server_key_version: "encryptionkey@0"
I20260806 18:40:10.618413 18629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "c6ff04d70057445c801791a432a44a1f"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "b67951fef0b68c9acc7f5fb6ae95f5ea"
server_key_iv: "2ec924b092e13246499a72eacd678cca"
server_key_version: "encryptionkey@0"
I20260806 18:40:10.622254 18629 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 18:40:10.624671 18663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:10.625887 18629 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:10.626013 18629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "c6ff04d70057445c801791a432a44a1f"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "b67951fef0b68c9acc7f5fb6ae95f5ea"
server_key_iv: "2ec924b092e13246499a72eacd678cca"
server_key_version: "encryptionkey@0"
I20260806 18:40:10.626107 18629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:10.644804 18629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:10.647890 18629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:10.648084 18629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:10.648659 18629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:10.649510 18629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:10.649559 18629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:10.649614 18629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:10.649641 18629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:10.658234 18629 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:44305
I20260806 18:40:10.658263 18852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:44305 every 8 connection(s)
I20260806 18:40:10.659122 18629 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:10.662060 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 18629
I20260806 18:40:10.662197 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:10.662544 9159 external_mini_cluster.cc:1428] Setting key 9c537bd4da9ca6b0e655759c84bfdfc0
Aug 06 18:40:10 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:40:10.673388 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:10.660812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:34859 (local address 127.8.241.254:43515)
0806 18:40:10.660938 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:10.660943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:10.661853 (+ 910us) server_negotiation.cc:407] Connection header received
0806 18:40:10.663371 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:10.663373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:10.663419 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:10.663493 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:10.664960 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.665502 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.666854 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.667205 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.667316 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:10.669892 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:10.669895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:10.669897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:10.669923 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:10.671257 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:10.671811 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:10.671815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:10.671817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:10.671895 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:10.672222 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:10.672226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:10.672228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:10.672412 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:10.672502 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:10.673075 (+ 573us) server_negotiation.cc:299] Negotiation successful
0806 18:40:10.673210 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":42}
I20260806 18:40:10.674317 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:10.661218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:34859)
0806 18:40:10.661685 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:10.661728 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:10.663165 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:10.663695 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:10.663704 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:10.664130 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:10.664800 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:10.664821 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.665622 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.665626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:10.666709 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:10.666721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.667345 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.667348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:10.667507 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:10.668040 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:10.668057 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:10.669728 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:10.671368 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:10.671375 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:10.671391 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:10.671723 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:10.671987 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:10.671990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:10.671992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:10.672134 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:10.672519 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:10.672526 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:10.672785 (+ 259us) client_negotiation.cc:769] Sending connection context
0806 18:40:10.672985 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 18:40:10.673361 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":104,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.194@KRBTEST.COM" created.
I20260806 18:40:10.675674 18853 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43515
I20260806 18:40:10.675930 18853 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:10.676517 18853 heartbeater.cc:511] Master 127.8.241.254:43515 requested a full tablet report, sending...
I20260806 18:40:10.678110 18507 ts_manager.cc:194] Registered new tserver with Master: c6ff04d70057445c801791a432a44a1f (127.8.241.193:44305)
I20260806 18:40:10.679268 18507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:34859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab.
Entry for principal oryx/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab.
I20260806 18:40:10.690425 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43515
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab
--principal=oryx/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:10.803900 18862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:10.804152 18862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:10.804195 18862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:10.808264 18862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:10.808323 18862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:10.808389 18862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:10.813665 18862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab
--principal=oryx/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:10.814760 18862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:10.815817 18862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:10.825452 18875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:10.825533 18870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:10.825453 18868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:10.825640 18862 server_base.cc:1034] running on GCE node
I20260806 18:40:10.826220 18862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:10.826911 18862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:10.828074 18862 hybrid_clock.cc:648] HybridClock initialized: now 1786041610828053 us; error 30 us; skew 500 ppm
Aug 06 18:40:10 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:10.830879 18862 init.cc:377] Logged in from keytab as oryx/127.8.241.194@KRBTEST.COM (short username oryx)
I20260806 18:40:10.831766 18862 webserver.cc:460] Webserver started at http://127.8.241.194:36033/ using document root <none> and password file <none>
I20260806 18:40:10.832319 18862 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:10.832381 18862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:10.832602 18862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:10.834692 18862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "87ac15e9acfb46c08395ce0aa8819ccc"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "e3ae06699caf549e617b4c6c262e5734"
server_key_iv: "c9e340bef3608f61447ef721d90da6ca"
server_key_version: "encryptionkey@0"
I20260806 18:40:10.835240 18862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "87ac15e9acfb46c08395ce0aa8819ccc"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "e3ae06699caf549e617b4c6c262e5734"
server_key_iv: "c9e340bef3608f61447ef721d90da6ca"
server_key_version: "encryptionkey@0"
I20260806 18:40:10.839071 18862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 18:40:10.841509 18888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:10.842703 18862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:10.842810 18862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "87ac15e9acfb46c08395ce0aa8819ccc"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "e3ae06699caf549e617b4c6c262e5734"
server_key_iv: "c9e340bef3608f61447ef721d90da6ca"
server_key_version: "encryptionkey@0"
I20260806 18:40:10.842903 18862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:10.851729 18862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:10.854822 18862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:10.855000 18862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:10.855582 18862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:10.856401 18862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:10.856457 18862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:10.856511 18862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:10.856534 18862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:10.864830 18862 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:39639
I20260806 18:40:10.864854 19060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:39639 every 8 connection(s)
I20260806 18:40:10.865732 18862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:10.867112 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 18862
I20260806 18:40:10.867218 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:40:10.867559 9159 external_mini_cluster.cc:1428] Setting key c9842c43b6857eb44b5166460c047d1e
Aug 06 18:40:10 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:40:10.879652 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:10.867515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:34289 (local address 127.8.241.254:43515)
0806 18:40:10.867666 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:10.867671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:10.868568 (+ 897us) server_negotiation.cc:407] Connection header received
0806 18:40:10.870019 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:10.870023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:10.870078 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:10.870172 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:10.871495 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.872045 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.873298 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.873608 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.873711 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:10.876355 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:10.876358 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:10.876360 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:10.876384 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:10.877676 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:10.878258 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:10.878260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:10.878261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:10.878326 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:10.878686 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:10.878689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:10.878690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:10.878867 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:10.878954 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:10.879392 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 18:40:10.879490 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:40:10.880492 19065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:10.867875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.194:34289)
0806 18:40:10.868408 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:10.868469 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:10.869748 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:10.870357 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:10.870365 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:10.870725 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:10.871344 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:10.871359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.872204 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.872207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:10.873165 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:10.873176 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:10.873736 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:10.873738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:10.873880 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:10.874453 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:10.874477 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:10.876181 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:10.877798 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:10.877804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:10.877821 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:10.878173 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:10.878403 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:10.878406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:10.878408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:10.878594 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:10.878965 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:10.878970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:10.879204 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 18:40:10.879363 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 18:40:10.879671 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.195@KRBTEST.COM" created.
I20260806 18:40:10.881762 19062 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43515
I20260806 18:40:10.882005 19062 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:10.882521 19062 heartbeater.cc:511] Master 127.8.241.254:43515 requested a full tablet report, sending...
I20260806 18:40:10.883630 18507 ts_manager.cc:194] Registered new tserver with Master: 87ac15e9acfb46c08395ce0aa8819ccc (127.8.241.194:39639)
I20260806 18:40:10.884367 18507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:34289
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab.
Entry for principal oryx/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab.
I20260806 18:40:10.896693 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43515
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab
--principal=oryx/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:11.011694 19070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:11.011970 19070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:11.012012 19070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:11.016047 19070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:11.016108 19070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:11.016165 19070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:11.021376 19070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab
--principal=oryx/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:11.022604 19070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:11.023694 19070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:11.033432 19076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.033481 19075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.033433 19078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:11.033708 19070 server_base.cc:1034] running on GCE node
I20260806 18:40:11.034197 19070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:11.034813 19070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:11.035974 19070 hybrid_clock.cc:648] HybridClock initialized: now 1786041611035950 us; error 35 us; skew 500 ppm
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:11.038712 19070 init.cc:377] Logged in from keytab as oryx/127.8.241.195@KRBTEST.COM (short username oryx)
I20260806 18:40:11.039618 19070 webserver.cc:460] Webserver started at http://127.8.241.195:43351/ using document root <none> and password file <none>
I20260806 18:40:11.040185 19070 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:11.040246 19070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:11.040452 19070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:11.042506 19070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "839cf65a241885f3670790cc3f64354f"
server_key_iv: "bdea70d1db5b2cdcf08fe27500bd34f6"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.043093 19070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "839cf65a241885f3670790cc3f64354f"
server_key_iv: "bdea70d1db5b2cdcf08fe27500bd34f6"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.046968 19070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:40:11.049407 19088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.050604 19070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:11.050704 19070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "839cf65a241885f3670790cc3f64354f"
server_key_iv: "bdea70d1db5b2cdcf08fe27500bd34f6"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.050789 19070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:11.062543 19070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:11.065302 19070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:11.065478 19070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:11.066038 19070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:11.066828 19070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:11.066874 19070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.066913 19070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:11.066933 19070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.075412 19070 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:37419
I20260806 18:40:11.075465 19277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:37419 every 8 connection(s)
I20260806 18:40:11.076290 19070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:11.083820 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 19070
I20260806 18:40:11.083930 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:11.084227 9159 external_mini_cluster.cc:1428] Setting key a9b6dc700e32afd94d2dbae6154e1f65
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:11.089303 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.078075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:47777 (local address 127.8.241.254:43515)
0806 18:40:11.078217 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.078222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.079180 (+ 958us) server_negotiation.cc:407] Connection header received
0806 18:40:11.080409 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.080412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.080460 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.080534 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.081712 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.082237 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.083289 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.083491 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.083563 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.086159 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.086162 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.086164 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.086188 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.087291 (+ 1103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.087802 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.087804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.087806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.087881 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.088197 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.088200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.088202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.088384 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.088471 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.088984 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.089141 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:40:11.090070 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.078488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.195:47777)
0806 18:40:11.079031 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.079074 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.080221 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.080692 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.080699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.081112 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.081599 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.081613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.082334 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.082336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.083200 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.083210 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.083596 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.083598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.083730 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.084385 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.084402 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.085999 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.087400 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.087406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.087421 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.087702 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.087965 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.087968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.087970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.088107 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.088494 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.088500 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.088740 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.088910 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.089254 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":152,"threads_started":1}
I20260806 18:40:11.091099 19278 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43515
I20260806 18:40:11.091311 19278 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:11.091779 19278 heartbeater.cc:511] Master 127.8.241.254:43515 requested a full tablet report, sending...
I20260806 18:40:11.092724 18507 ts_manager.cc:194] Registered new tserver with Master: 79bd32d76dd24d43b2bc5aacc4e72f3b (127.8.241.195:37419)
I20260806 18:40:11.093305 18507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:47777
I20260806 18:40:11.097821 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.241.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.253.keytab.
Entry for principal oryx/127.8.241.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.253.keytab.
I20260806 18:40:11.123343 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.253:41923
--webserver_interface=127.8.241.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.253.keytab
--principal=oryx/127.8.241.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:43515,127.8.241.253:41923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:11.235181 19287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:11.235445 19287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:11.235502 19287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:11.239476 19287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:11.239533 19287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:11.239559 19287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:11.239583 19287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:11.239611 19287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:11.244827 19287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:43515,127.8.241.253:41923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.253.keytab
--principal=oryx/127.8.241.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.253:41923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.241.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
W20260806 18:40:11.245061 19287 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.241.254:43515,127.8.241.253:41923'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 18:40:11.245945 19287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:11.247048 19287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:11.255425 19296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.255434 19300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:11.255612 19287 server_base.cc:1034] running on GCE node
W20260806 18:40:11.255434 19294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:11.256290 19287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:11.257450 19287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:11.258625 19287 hybrid_clock.cc:648] HybridClock initialized: now 1786041611258593 us; error 41 us; skew 500 ppm
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:11.261224 19287 init.cc:377] Logged in from keytab as oryx/127.8.241.253@KRBTEST.COM (short username oryx)
I20260806 18:40:11.262117 19287 webserver.cc:460] Webserver started at http://127.8.241.253:43019/ using document root <none> and password file <none>
I20260806 18:40:11.262661 19287 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:11.262727 19287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:11.262940 19287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:11.264918 19287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data/instance:
uuid: "36971c9b70d741a58fecf5e5690873f0"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "403d4275a525db3108e1cdf2e8fd555e"
server_key_iv: "3d021da03f253f6119d2cb91df48c3e3"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.265488 19287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal/instance:
uuid: "36971c9b70d741a58fecf5e5690873f0"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "403d4275a525db3108e1cdf2e8fd555e"
server_key_iv: "3d021da03f253f6119d2cb91df48c3e3"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.269096 19287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:40:11.271556 19314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.272717 19287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:11.272835 19287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal
uuid: "36971c9b70d741a58fecf5e5690873f0"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "403d4275a525db3108e1cdf2e8fd555e"
server_key_iv: "3d021da03f253f6119d2cb91df48c3e3"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.272927 19287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:11.292029 19287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:11.294948 19287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:11.295131 19287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:11.301906 19287 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.253:41923
I20260806 18:40:11.302168 19414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.253:41923 every 8 connection(s)
I20260806 18:40:11.302886 19287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
I20260806 18:40:11.305824 19416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:11.309134 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 19287
I20260806 18:40:11.309818 19416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.253@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.241.253@KRBTEST.COM
I20260806 18:40:11.321228 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.311759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57644 (local address 127.8.241.254:43515)
0806 18:40:11.311875 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.311879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.312193 (+ 314us) server_negotiation.cc:407] Connection header received
0806 18:40:11.312515 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.312518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.312570 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.312637 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.313475 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.314015 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.314996 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.315206 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.315292 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.317813 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.317817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.317819 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.317843 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.319245 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.319746 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.319750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.319752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.319829 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.320135 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.320138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.320140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.320352 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.320440 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.320912 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.321060 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:40:11.322073 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.311713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57644)
0806 18:40:11.312069 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.312106 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.312344 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.312698 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.312704 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.313076 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.313335 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.313349 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.314139 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.314141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.314878 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.314887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.315346 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.315350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.315486 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.316192 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.316208 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.317660 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.319352 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.319361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.319372 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.319653 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.319911 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.319915 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.319916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.320048 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.320463 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.320469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.320702 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.320851 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.321179 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":46,"spinlock_wait_cycles":3456,"thread_start_us":129,"threads_started":1}
I20260806 18:40:11.322077 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.310680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42936 (local address 127.8.241.253:41923)
0806 18:40:11.311097 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.311111 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.311143 (+ 32us) server_negotiation.cc:407] Connection header received
0806 18:40:11.311973 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.311997 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.312344 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.312567 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.313384 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.314242 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.315028 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.315378 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.315494 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.318460 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.318465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.318476 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.318496 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.319800 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.320177 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.320182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.320183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.320256 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.320499 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.320503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.320504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.320746 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.320930 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.321156 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.321287 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":280,"thread_start_us":122,"threads_started":1}
I20260806 18:40:11.323401 19416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.253@KRBTEST.COM for oryx/127.8.241.253@KRBTEST.COM
I20260806 18:40:11.330849 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.323888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.8.241.253:41923)
0806 18:40:11.323980 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.323983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.323994 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:40:11.324101 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.324104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.324144 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.324215 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.324623 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.325134 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.325811 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.326588 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.326669 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.328333 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.328336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.328338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.328358 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.329528 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.329921 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.329923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.329924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.329991 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.330222 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.330224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.330226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.330379 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.330460 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.330638 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.330745 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":35}
I20260806 18:40:11.330850 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.323847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.0.0.1:42952)
0806 18:40:11.323940 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.323951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.324018 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.324344 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.324346 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.324461 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.324543 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.324548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.325221 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.325225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.325729 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.325738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.326673 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.326675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.326739 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.327121 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.327132 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.328220 (+ 1088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.329633 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.329637 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.329639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.329846 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.330056 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.330058 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.330060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.330142 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.330450 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.330452 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.330522 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.330598 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.330695 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 18:40:11.333567 19416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Bootstrap starting.
I20260806 18:40:11.335733 19416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:11.336426 19416 log.cc:824] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:11.338248 19416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: No bootstrap required, opened a new log
I20260806 18:40:11.341259 19416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } }
I20260806 18:40:11.341480 19416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:11.341528 19416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36971c9b70d741a58fecf5e5690873f0, State: Initialized, Role: FOLLOWER
I20260806 18:40:11.341929 19416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } }
I20260806 18:40:11.342362 19428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } } }
I20260806 18:40:11.342475 19428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:40:11.343036 19416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:40:11.347824 19447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:40:11.347903 19447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:40:11.350711 19287 master_runner.cc:428] Detected that this master 36971c9b70d741a58fecf5e5690873f0 is joining an existing cluster
I20260806 18:40:11.350766 19287 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.241.253:41923
I20260806 18:40:11.362210 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.353505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42954 (local address 127.8.241.253:41923)
0806 18:40:11.353592 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.353596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.353968 (+ 372us) server_negotiation.cc:407] Connection header received
0806 18:40:11.354111 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.354114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.354145 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.354186 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.354792 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.355561 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.356797 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.357126 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.357185 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.358441 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.358445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.358456 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.358482 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.360591 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.361117 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.361120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.361122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.361197 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.361418 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.361422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.361424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.361622 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.361703 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.361957 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.362087 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":28}
I20260806 18:40:11.362221 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.353447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.0.0.1:42954)
0806 18:40:11.353898 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.353910 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.354027 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.354303 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.354306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.354562 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.354682 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.354690 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.355705 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.355708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.356696 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.356705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.357322 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.357326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.357405 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.357906 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.357918 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.358310 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.360700 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.360703 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.360705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.360996 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.361281 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.361284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.361286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.361351 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.361702 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.361704 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.361786 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.361900 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.362035 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":69,"thread_start_us":81,"threads_started":1}
I20260806 18:40:11.362581 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.353717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.8.241.254:43515)
0806 18:40:11.353847 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.353851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.353865 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:11.353907 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.353910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.353957 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.354031 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.354802 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.355604 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.356865 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.357222 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.357311 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.359060 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.359064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.359067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.359091 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.360624 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.361182 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.361185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.361187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.361270 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.361553 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.361557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.361559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.361897 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.361991 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.362299 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.362420 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:40:11.363014 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.353027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57654)
0806 18:40:11.353325 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.353340 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.353431 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.354153 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.354156 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.354438 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.354593 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.354601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.355748 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.355751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.356714 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.356727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.357649 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.357652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.357732 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.358466 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.358494 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.358925 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.360737 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.360740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.360742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.361065 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.361352 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.361355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.361357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.361452 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.362036 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.362039 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.362127 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.362704 (+ 577us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.362837 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":5,"spinlock_wait_cycles":9344,"thread_start_us":105,"threads_started":1}
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.254@KRBTEST.COM for oryx/127.8.241.253@KRBTEST.COM
I20260806 18:40:11.375604 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.367125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42960 (local address 127.8.241.253:41923)
0806 18:40:11.367229 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.367233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.367512 (+ 279us) server_negotiation.cc:407] Connection header received
0806 18:40:11.367708 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.367711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.367783 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.367855 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.368390 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.369185 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.370042 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.370367 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.370477 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.372459 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.372468 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.372471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.372496 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.373917 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.374329 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.374333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.374335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.374417 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.374696 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.374700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.374702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.374884 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.374981 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.375346 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.375466 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:40:11.375674 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.367065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.0.0.1:42960)
0806 18:40:11.367399 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.367443 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.367571 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.367887 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.367892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.368203 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.368295 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.368300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.369275 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.369277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.369877 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.369885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.370468 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.370471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.370545 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.370982 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.370994 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.372327 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.374015 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.374024 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.374029 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.374249 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.374515 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.374518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.374520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.374606 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.375000 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.375007 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.375142 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.375293 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.375500 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
I20260806 18:40:11.377313 18507 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.241.253:41923: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "36971c9b70d741a58fecf5e5690873f0"
member_type: NON_VOTER
last_known_addr {
host: "127.8.241.253"
port: 41923
}
attrs {
promote: true
}
}
dest_uuid: "7bc7173999834375bfaeaf8f77292c24"
cas_config_opid_index: -1
I20260806 18:40:11.378048 18507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } }
I20260806 18:40:11.379910 19465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 36971c9b70d741a58fecf5e5690873f0 (127.8.241.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } } }
I20260806 18:40:11.380805 19367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:11.382194 19367 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Refusing update from remote peer 7bc7173999834375bfaeaf8f77292c24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 18:40:11.382227 19465 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.241.253:41923
I20260806 18:40:11.382843 19287 master.cc:537] Master@127.8.241.253:41923 shutting down...
I20260806 18:40:11.383154 19466 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
WARNING: no policy specified for oryx/127.8.241.252@KRBTEST.COM; defaulting to no policy
W20260806 18:40:11.385056 18484 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 18:40:11.385341 19465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } } }
I20260806 18:40:11.385542 19465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
W20260806 18:40:11.386227 18484 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 -> Peer 36971c9b70d741a58fecf5e5690873f0 (127.8.241.253:41923): Couldn't send request to peer 36971c9b70d741a58fecf5e5690873f0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:40:11.387121 19287 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:11.387390 19287 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:11.387506 19287 tablet_replica.cc:330] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.252@KRBTEST.COM" created.
I20260806 18:40:11.392246 19287 master.cc:552] Master@127.8.241.253:41923 shutdown complete.
I20260806 18:40:11.392297 19287 master_runner.cc:305] Clearing existing system tablet
I20260806 18:40:11.392540 19287 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:11.392593 19287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:11.393929 19287 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 18:40:11.394801 19475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.395040 19287 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:11.395160 19287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:11.395192 19287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal
uuid: "36971c9b70d741a58fecf5e5690873f0"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "403d4275a525db3108e1cdf2e8fd555e"
server_key_iv: "3d021da03f253f6119d2cb91df48c3e3"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.397363 19287 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:11.399686 19287 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:40:11.399775 19287 log.cc:1196] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 18:40:11.400005 19287 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Deleting consensus metadata
I20260806 18:40:11.400156 19287 master_runner.cc:315] Copying system tablet from 127.8.241.254:43515
I20260806 18:40:11.401341 19287 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: tablet copy: Beginning tablet copy session from remote peer at address 127.8.241.254:43515
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.252.keytab.
Entry for principal oryx/127.8.241.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.252.keytab.
I20260806 18:40:11.404633 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.252:33933
--webserver_interface=127.8.241.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.252.keytab
--principal=oryx/127.8.241.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:43515,127.8.241.253:41923,127.8.241.252:33933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:40:11.410471 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.402015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.8.241.254:43515)
0806 18:40:11.402145 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.402149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.402261 (+ 112us) server_negotiation.cc:407] Connection header received
0806 18:40:11.402428 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.402431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.402484 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.402575 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.403025 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.403841 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.405046 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.405382 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.405473 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.406903 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.406908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.406911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.406939 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.408475 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.408993 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.408997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.408998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.409126 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.409468 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.409472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.409474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.409703 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.409779 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.410163 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.410288 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:40:11.411058 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.401916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57664)
0806 18:40:11.402172 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.402187 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.402329 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.402594 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.402596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.402791 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.402905 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.402911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.403977 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.403981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.404886 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.404897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.405635 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.405638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.405732 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.406328 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.406344 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.406751 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.408599 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.408602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.408604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.408874 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.409249 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.409252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.409254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.409351 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.409858 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.409861 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.409966 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.410706 (+ 740us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.410862 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":98,"threads_started":1}
I20260806 18:40:11.411760 18546 tablet_copy_service.cc:140] P 7bc7173999834375bfaeaf8f77292c24: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 36971c9b70d741a58fecf5e5690873f0 ({username='oryx', principal='oryx/127.8.241.253@KRBTEST.COM'} at 127.0.0.1:57664)
I20260806 18:40:11.411966 18546 tablet_copy_service.cc:161] P 7bc7173999834375bfaeaf8f77292c24: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 36971c9b70d741a58fecf5e5690873f0 at {username='oryx', principal='oryx/127.8.241.253@KRBTEST.COM'} at 127.0.0.1:57664: session id = 36971c9b70d741a58fecf5e5690873f0-00000000000000000000000000000000
I20260806 18:40:11.413957 18546 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:40:11.420550 19287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:40:11.423149 19287 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: tablet copy: Starting download of 0 data blocks...
I20260806 18:40:11.423542 19287 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: tablet copy: Starting download of 1 WAL segments...
I20260806 18:40:11.426271 19287 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:40:11.428176 18546 tablet_copy_service.cc:342] P 7bc7173999834375bfaeaf8f77292c24: Request end of tablet copy session 36971c9b70d741a58fecf5e5690873f0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.241.253@KRBTEST.COM'} at 127.0.0.1:57664
I20260806 18:40:11.428315 18546 tablet_copy_service.cc:433] P 7bc7173999834375bfaeaf8f77292c24: ending tablet copy session 36971c9b70d741a58fecf5e5690873f0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 36971c9b70d741a58fecf5e5690873f0
I20260806 18:40:11.437249 19287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:11.441195 19493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.457187 19494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.457561 19497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:11.458139 19287 server_base.cc:1034] running on GCE node
I20260806 18:40:11.458317 19287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:11.458642 19287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:11.459873 19287 hybrid_clock.cc:648] HybridClock initialized: now 1786041611459860 us; error 30 us; skew 500 ppm
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:11.461711 19287 init.cc:377] Logged in from keytab as oryx/127.8.241.253@KRBTEST.COM (short username oryx)
I20260806 18:40:11.462232 19287 webserver.cc:460] Webserver started at http://127.8.241.253:42589/ using document root <none> and password file <none>
I20260806 18:40:11.462487 19287 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:11.462576 19287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:11.463965 19287 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:11.465041 19506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.465308 19287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:11.465382 19287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal
uuid: "36971c9b70d741a58fecf5e5690873f0"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "403d4275a525db3108e1cdf2e8fd555e"
server_key_iv: "3d021da03f253f6119d2cb91df48c3e3"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.465487 19287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:11.478664 19287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:11.480676 19287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:11.485873 19287 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.253:41923
I20260806 18:40:11.485910 19593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.253:41923 every 8 connection(s)
I20260806 18:40:11.486475 19287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
I20260806 18:40:11.488330 19594 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:40:11.489439 19594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Bootstrap starting.
I20260806 18:40:11.504204 19594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:40:11.504508 19594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Bootstrap complete.
I20260806 18:40:11.505116 19594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } }
I20260806 18:40:11.505249 19594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 36971c9b70d741a58fecf5e5690873f0, State: Initialized, Role: LEARNER
I20260806 18:40:11.505350 19594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } }
I20260806 18:40:11.505798 19594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:11.513262 19599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } } }
I20260806 18:40:11.513427 19599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:40:11.518074 19615 catalog_manager.cc:1259] Loaded cluster ID: e2853932c97c4ac2b942f359e8756eb2
I20260806 18:40:11.518131 19615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: loading cluster ID for follower catalog manager: success
I20260806 18:40:11.519765 19615 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: acquiring CA information for follower catalog manager: success
I20260806 18:40:11.520862 19615 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 18:40:11.557848 19492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:11.558096 19492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:11.558141 19492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:11.562007 19492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:11.562057 19492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:11.562075 19492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:11.562104 19492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:11.562132 19492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:11.568075 19492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:43515,127.8.241.253:41923,127.8.241.252:33933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.252.keytab
--principal=oryx/127.8.241.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.252:33933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.241.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:11.569151 19492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:11.570202 19492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:11.579615 19622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.579734 19624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.579622 19621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:11.582892 19492 server_base.cc:1034] running on GCE node
I20260806 18:40:11.583585 19492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:11.584867 19492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:11.586030 19492 hybrid_clock.cc:648] HybridClock initialized: now 1786041611586008 us; error 38 us; skew 500 ppm
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:11.588655 19492 init.cc:377] Logged in from keytab as oryx/127.8.241.252@KRBTEST.COM (short username oryx)
I20260806 18:40:11.589632 19492 webserver.cc:460] Webserver started at http://127.8.241.252:44765/ using document root <none> and password file <none>
I20260806 18:40:11.590200 19492 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:11.590267 19492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:11.590487 19492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:11.592645 19492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data/instance:
uuid: "eb579dc4a65a4555b6c1011b3a874f74"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "100dbb40eaaf0029dc80e83840d5cb50"
server_key_iv: "3d42f70daa2eecec0cf475dd76867e57"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.593226 19492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal/instance:
uuid: "eb579dc4a65a4555b6c1011b3a874f74"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "100dbb40eaaf0029dc80e83840d5cb50"
server_key_iv: "3d42f70daa2eecec0cf475dd76867e57"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.596972 19492 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:40:11.599396 19635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.600598 19492 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:11.600704 19492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal
uuid: "eb579dc4a65a4555b6c1011b3a874f74"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "100dbb40eaaf0029dc80e83840d5cb50"
server_key_iv: "3d42f70daa2eecec0cf475dd76867e57"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.600792 19492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:11.618505 19492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:11.621170 19492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:11.621322 19492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:11.628270 19492 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.252:33933
I20260806 18:40:11.628513 19717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.252:33933 every 8 connection(s)
I20260806 18:40:11.629288 19492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
I20260806 18:40:11.632189 19719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:11.633869 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 19492
I20260806 18:40:11.636482 19719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.241.252@KRBTEST.COM
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.252@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:11.647799 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.638272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.8.241.254:43515)
0806 18:40:11.638399 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.638404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.638605 (+ 201us) server_negotiation.cc:407] Connection header received
0806 18:40:11.638748 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.638751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.638795 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.638862 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.639633 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.640431 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.641569 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.641830 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.641933 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.644373 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.644376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.644378 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.644401 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.645954 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.646463 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.646466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.646468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.646546 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.646819 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.646821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.646822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.647015 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.647115 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.647527 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.647635 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:40:11.648509 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.635334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49128 (local address 127.8.241.252:33933)
0806 18:40:11.635754 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.635770 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.635805 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:40:11.636452 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.636469 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.636796 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.637171 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.637992 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.639458 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.640636 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.641335 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.641492 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.643876 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.643883 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.643897 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.643923 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.645707 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.646186 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.646192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.646194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.646298 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.646591 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.646594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.646596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.647090 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.647301 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.647515 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.647758 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":118,"threads_started":1}
I20260806 18:40:11.649057 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.638080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57680)
0806 18:40:11.638473 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.638522 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.638646 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.638924 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.638929 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.639342 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.639532 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.639540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.640537 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.640540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.641463 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.641473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.641999 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.642004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.642096 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.642526 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.642542 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.644228 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.646059 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.646067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.646078 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.646366 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.646650 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.646653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.646655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.646736 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.647123 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.647130 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.647294 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.647474 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.648857 (+ 1383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":1115,"thread_start_us":126,"threads_started":1}
I20260806 18:40:11.650151 19719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.252@KRBTEST.COM for oryx/127.8.241.253@KRBTEST.COM
I20260806 18:40:11.657466 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.650607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.0.0.1:42976)
0806 18:40:11.650755 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.650768 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.650839 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.651164 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.651166 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.651285 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.651375 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.651380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.652178 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.652180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.652689 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.652696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.653130 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.653132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.653206 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.653549 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.653559 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.654709 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.656311 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.656313 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.656315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.656492 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.656691 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.656692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.656694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.656757 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.657103 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.657106 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.657181 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.657261 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.657354 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 18:40:11.657547 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.650723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.8.241.253:41923)
0806 18:40:11.650985 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.650988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.651000 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:40:11.651035 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.651037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.651076 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.651145 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.651467 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.652106 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.652785 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.653039 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.653123 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.654828 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.654831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.654833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.654856 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.656213 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.656562 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.656564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.656565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.656632 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.656812 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.656814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.656815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.656978 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.657105 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.657305 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.657400 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":105,"threads_started":1}
I20260806 18:40:11.658259 19719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.252@KRBTEST.COM for oryx/127.8.241.252@KRBTEST.COM
I20260806 18:40:11.665457 19725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.658708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.0.0.1:49144)
0806 18:40:11.658807 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.658820 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.658888 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.659091 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.659093 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.659239 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.659301 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.659306 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.660375 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.660377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.660854 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.660861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.661331 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.661334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.661456 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.661863 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.661873 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.662985 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.664242 (+ 1257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.664244 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.664245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.664423 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.664633 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.664635 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.664636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.664703 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.664983 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.664985 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.665091 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.665176 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.665303 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":17}
I20260806 18:40:11.665473 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.658840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.8.241.252:33933)
0806 18:40:11.658940 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.658944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.658955 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:40:11.658982 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.658985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.659018 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.659080 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.659398 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.659898 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.660962 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.661235 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.661323 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.663097 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.663100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.663102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.663123 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.664152 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.664504 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.664506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.664508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.664570 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.664762 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.664764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.664765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.664907 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.664987 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.665233 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.665353 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":32}
I20260806 18:40:11.668308 19719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: Bootstrap starting.
I20260806 18:40:11.670775 19719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:11.671499 19719 log.cc:824] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:11.673333 19719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: No bootstrap required, opened a new log
I20260806 18:40:11.676347 19719 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } }
I20260806 18:40:11.676563 19719 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:11.676622 19719 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb579dc4a65a4555b6c1011b3a874f74, State: Initialized, Role: FOLLOWER
I20260806 18:40:11.677099 19719 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } }
I20260806 18:40:11.677591 19732 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } } }
I20260806 18:40:11.677709 19732 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:40:11.678318 19719 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:11.681856 18853 heartbeater.cc:503] Master 127.8.241.254:43515 was elected leader, sending a full tablet report...
W20260806 18:40:11.682799 19754 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:40:11.682875 19754 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:40:11.686136 19492 master_runner.cc:428] Detected that this master eb579dc4a65a4555b6c1011b3a874f74 is joining an existing cluster
I20260806 18:40:11.686185 19492 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.241.252:33933
I20260806 18:40:11.701719 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 18629
I20260806 18:40:11.727723 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.688242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42978 (local address 127.8.241.253:41923)
0806 18:40:11.688364 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.688368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.688980 (+ 612us) server_negotiation.cc:407] Connection header received
0806 18:40:11.689185 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.689188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.689237 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.689286 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.690014 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.690804 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.692102 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.692434 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.692523 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.723281 (+ 30758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.723285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.723288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.723319 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.725046 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.725909 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.725914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.725918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.726036 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.726434 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.726439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.726441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.726784 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.726877 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.727334 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.727493 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:40:11.728606 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.688477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.0.0.1:42978)
0806 18:40:11.688894 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.688906 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.689073 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.689425 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.689428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.689721 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.689873 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.689882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.690947 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.690950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.691968 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.691980 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.692675 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.692678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.692772 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.722450 (+ 29678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.722473 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.723071 (+ 598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.725187 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.725191 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.725193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.725750 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.726185 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.726189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.726191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.726298 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.726968 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.726971 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.727093 (+ 122us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.728196 (+ 1103us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.728382 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":106,"thread_start_us":100,"threads_started":1}
I20260806 18:40:11.729357 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.688163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.8.241.254:43515)
0806 18:40:11.688284 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.688288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.688522 (+ 234us) server_negotiation.cc:407] Connection header received
0806 18:40:11.688711 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.688715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.688767 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.688818 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.689505 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.690289 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.691508 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.691826 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.691894 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.725365 (+ 33471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.725370 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.725373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.725406 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.727279 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.727801 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.727806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.727808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.727910 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.728273 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.728277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.728279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.728521 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.728606 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.729039 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.729173 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:40:11.729465 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:44305
--local_ip_for_outbound_sockets=127.8.241.193
--tserver_master_addrs=127.8.241.254:43515
--webserver_port=35227
--webserver_interface=127.8.241.193
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab
--principal=oryx/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.241.254:43515,127.8.241.253:41923
--tserver_master_addrs=127.8.241.254:43515,127.8.241.253:41923,127.8.241.252:33933 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:40:11.729923 19764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.688114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57690)
0806 18:40:11.688446 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.688461 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.688595 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.688984 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.688987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.689242 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.689383 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.689390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.690417 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.690420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.691360 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.691371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.692073 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.692076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.692150 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.692618 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.692635 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.724178 (+ 31543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.727421 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.727424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.727426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.727662 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.728049 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.728052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.728054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.728142 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.728694 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.728697 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.728808 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.729581 (+ 773us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.729735 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":23,"thread_start_us":95,"threads_started":1}
I20260806 18:40:11.732980 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.688906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.8.241.252:33933)
0806 18:40:11.689001 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.689027 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.689211 (+ 184us) server_negotiation.cc:407] Connection header received
0806 18:40:11.689340 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.689343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.689375 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.689415 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.690003 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.690661 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.691661 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.692009 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.692073 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.723259 (+ 31186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.723263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.723266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.723289 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.725320 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.731645 (+ 6325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.731649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.731651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.731759 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.732056 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.732059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.732061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.732273 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.732364 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.732691 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.732834 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":39}
I20260806 18:40:11.733733 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.688602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.0.0.1:49152)
0806 18:40:11.689146 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.689158 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.689259 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.689530 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.689533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.689764 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.689898 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.689905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.690766 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.690769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.691566 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.691575 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.692194 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.692197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.692273 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.692948 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.692962 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.723071 (+ 30109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.729122 (+ 6051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.729125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.729127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.729366 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.731872 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.731875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.731877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.731962 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.732417 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.732420 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.732512 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.733402 (+ 890us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.733544 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041610, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.254@KRBTEST.COM for oryx/127.8.241.252@KRBTEST.COM
I20260806 18:40:11.742039 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.734701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59868 (local address 127.8.241.252:33933)
0806 18:40:11.734819 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.734824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.734837 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:11.734905 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.734909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.734963 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.735038 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.735541 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.736027 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.736794 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.737000 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.737142 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.739054 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.739057 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.739059 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.739081 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.740374 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.740803 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.740808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.740809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.740886 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.741232 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.741236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.741238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.741411 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.741507 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.741779 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.741879 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:40:11.742043 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.734086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.0.0.1:59868)
0806 18:40:11.734396 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.734412 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.734495 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.735166 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.735170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.735333 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.735422 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.735428 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.736135 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.736139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.736677 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.736685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.737137 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.737141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.737250 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.737670 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.737683 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.738913 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.740492 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.740495 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.740498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.740699 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.740975 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.740978 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.740980 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.741122 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.741510 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.741513 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.741615 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.741720 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.741878 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229}
I20260806 18:40:11.743323 18507 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.241.252:33933: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74"
member_type: NON_VOTER
last_known_addr {
host: "127.8.241.252"
port: 33933
}
attrs {
promote: true
}
}
dest_uuid: "7bc7173999834375bfaeaf8f77292c24"
cas_config_opid_index: 5
I20260806 18:40:11.743966 18507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } }
I20260806 18:40:11.745065 19465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.746371 19680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:11.746585 19465 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.241.252:33933
I20260806 18:40:11.746779 19465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.746888 19465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:11.747491 19492 master.cc:537] Master@127.8.241.252:33933 shutting down...
I20260806 18:40:11.747802 19680 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Refusing update from remote peer 7bc7173999834375bfaeaf8f77292c24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:40:11.748463 19466 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:40:11.750386 18482 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 -> Peer eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252:33933): Couldn't send request to peer eb579dc4a65a4555b6c1011b3a874f74. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:40:11.751583 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.745235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.0.0.1:41834)
0806 18:40:11.746149 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.746165 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.746250 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.747333 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.747336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.747357 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:11.747449 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.747457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.748658 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.748662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.749971 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.749984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.751167 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.751171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.751268 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.751320 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.751397 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.751486 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845}
I20260806 18:40:11.752017 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.745492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.8.241.253:41923)
0806 18:40:11.746054 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.746060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.746341 (+ 281us) server_negotiation.cc:407] Connection header received
0806 18:40:11.746402 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.746405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.746475 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.746579 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:11.747630 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.748359 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.750129 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.750746 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.750861 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.751028 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.751847 (+ 819us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.751924 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473}
I20260806 18:40:11.752692 19541 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 LEARNER]: Refusing update from remote peer 7bc7173999834375bfaeaf8f77292c24: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:40:11.753198 19466 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 18:40:11.753278 19492 raft_consensus.cc:2229] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:11.753721 19492 raft_consensus.cc:2258] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:11.753906 19492 tablet_replica.cc:330] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: stopping tablet replica
I20260806 18:40:11.755323 19541 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.757113 19599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.757282 19599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:40:11.758742 19466 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: attempting to promote NON_VOTER 36971c9b70d741a58fecf5e5690873f0 to VOTER
I20260806 18:40:11.759258 19466 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } }
I20260806 18:40:11.757090 19774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bc7173999834375bfaeaf8f77292c24. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: NON_VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: true } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.759997 19774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:40:11.760722 19541 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 LEARNER]: Refusing update from remote peer 7bc7173999834375bfaeaf8f77292c24: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 18:40:11.761056 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.760701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.0.0.1:59876)
0806 18:40:11.760875 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.760955 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:33933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 18:40:11.761219 19465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 18:40:11.761952 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.761527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.0.0.1:59886)
0806 18:40:11.761821 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.761874 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:33933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":111,"threads_started":1}
W20260806 18:40:11.762042 18482 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 -> Peer eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252:33933): Couldn't send request to peer eb579dc4a65a4555b6c1011b3a874f74. Status: Network error: Client connection negotiation failed: client connection to 127.8.241.252:33933: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 18:40:11.762989 19465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 36971c9b70d741a58fecf5e5690873f0 (127.8.241.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.764186 19465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.764313 19465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:11.764483 19465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.764559 19465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:11.765096 19541 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 36971c9b70d741a58fecf5e5690873f0 (127.8.241.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.766438 19774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.766577 19774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:40:11.771404 19492 master.cc:552] Master@127.8.241.252:33933 shutdown complete.
I20260806 18:40:11.771476 19492 master_runner.cc:305] Clearing existing system tablet
I20260806 18:40:11.771716 19492 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:11.771772 19492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:11.772903 19492 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 18:40:11.773756 19782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.773972 19492 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:11.774106 19492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:40:11.774149 19492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal
uuid: "eb579dc4a65a4555b6c1011b3a874f74"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "100dbb40eaaf0029dc80e83840d5cb50"
server_key_iv: "3d42f70daa2eecec0cf475dd76867e57"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.776001 19492 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:11.778105 19492 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:40:11.778201 19492 log.cc:1196] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 18:40:11.778436 19492 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: Deleting consensus metadata
I20260806 18:40:11.778601 19492 master_runner.cc:315] Copying system tablet from 127.8.241.254:43515
I20260806 18:40:11.779757 19492 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: tablet copy: Beginning tablet copy session from remote peer at address 127.8.241.254:43515
I20260806 18:40:11.787770 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.780296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57378)
0806 18:40:11.780638 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.780650 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.780780 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.781079 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.781082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.781292 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.781400 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.781407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.782167 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.782171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.782767 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.782776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.783227 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.783232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.783329 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.783820 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.783836 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.784169 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.786063 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.786067 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.786069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.786304 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.786607 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.786611 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.786613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.786717 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.787168 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.787172 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.787270 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.787373 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.787520 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":92,"threads_started":1}
I20260806 18:40:11.788077 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.780378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.8.241.254:43515)
0806 18:40:11.780509 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.780515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.780735 (+ 220us) server_negotiation.cc:407] Connection header received
0806 18:40:11.780893 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.780896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.780948 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.781048 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.781525 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.782055 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.782887 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.783116 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.783221 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.784316 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.784321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.784324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.784352 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.785939 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.786399 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.786403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.786405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.786498 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.786809 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.786813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.786815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.787028 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.787127 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.787474 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.787591 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:40:11.788431 18546 tablet_copy_service.cc:140] P 7bc7173999834375bfaeaf8f77292c24: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer eb579dc4a65a4555b6c1011b3a874f74 ({username='oryx', principal='oryx/127.8.241.252@KRBTEST.COM'} at 127.0.0.1:57378)
I20260806 18:40:11.788535 18546 tablet_copy_service.cc:161] P 7bc7173999834375bfaeaf8f77292c24: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer eb579dc4a65a4555b6c1011b3a874f74 at {username='oryx', principal='oryx/127.8.241.252@KRBTEST.COM'} at 127.0.0.1:57378: session id = eb579dc4a65a4555b6c1011b3a874f74-00000000000000000000000000000000
I20260806 18:40:11.789458 18546 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:40:11.790617 19492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:40:11.792505 19492 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: tablet copy: Starting download of 0 data blocks...
I20260806 18:40:11.792690 19492 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: tablet copy: Starting download of 1 WAL segments...
I20260806 18:40:11.794137 19492 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:40:11.795655 18546 tablet_copy_service.cc:342] P 7bc7173999834375bfaeaf8f77292c24: Request end of tablet copy session eb579dc4a65a4555b6c1011b3a874f74-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.241.252@KRBTEST.COM'} at 127.0.0.1:57378
I20260806 18:40:11.795739 18546 tablet_copy_service.cc:433] P 7bc7173999834375bfaeaf8f77292c24: ending tablet copy session eb579dc4a65a4555b6c1011b3a874f74-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer eb579dc4a65a4555b6c1011b3a874f74
I20260806 18:40:11.802762 19492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:11.804312 19792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.804454 19798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:11.804733 19492 server_base.cc:1034] running on GCE node
W20260806 18:40:11.805128 19795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:11.805349 19492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:11.805641 19492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:11.806779 19492 hybrid_clock.cc:648] HybridClock initialized: now 1786041611806769 us; error 30 us; skew 500 ppm
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:11.808303 19492 init.cc:377] Logged in from keytab as oryx/127.8.241.252@KRBTEST.COM (short username oryx)
I20260806 18:40:11.808784 19492 webserver.cc:460] Webserver started at http://127.8.241.252:40261/ using document root <none> and password file <none>
I20260806 18:40:11.808979 19492 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:11.809042 19492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:11.810225 19492 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:40:11.811106 19812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.811331 19492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 18:40:11.811409 19492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal
uuid: "eb579dc4a65a4555b6c1011b3a874f74"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "100dbb40eaaf0029dc80e83840d5cb50"
server_key_iv: "3d42f70daa2eecec0cf475dd76867e57"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.811513 19492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:11.827812 19492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:11.829847 19492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:11.835013 19492 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.252:33933
I20260806 18:40:11.835034 19882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.252:33933 every 8 connection(s)
I20260806 18:40:11.835593 19492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
I20260806 18:40:11.837241 19883 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:40:11.838196 19883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: Bootstrap starting.
I20260806 18:40:11.848227 19883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:40:11.848500 19883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: Bootstrap complete.
I20260806 18:40:11.849046 19883 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } }
I20260806 18:40:11.849177 19883 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eb579dc4a65a4555b6c1011b3a874f74, State: Initialized, Role: LEARNER
I20260806 18:40:11.849306 19883 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } }
I20260806 18:40:11.849632 19893 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:11.849793 19883 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:11.849833 19893 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:40:11.853714 19911 catalog_manager.cc:1259] Loaded cluster ID: e2853932c97c4ac2b942f359e8756eb2
I20260806 18:40:11.853842 19911 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: loading cluster ID for follower catalog manager: success
I20260806 18:40:11.855543 19911 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: acquiring CA information for follower catalog manager: success
I20260806 18:40:11.856691 19911 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:40:11.886487 19062 heartbeater.cc:503] Master 127.8.241.254:43515 was elected leader, sending a full tablet report...
W20260806 18:40:11.893888 19770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:11.894158 19770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:11.894201 19770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:11.898186 19770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:11.898247 19770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:11.898303 19770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:11.903518 19770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab
--principal=oryx/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:44305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=35227
--tserver_master_addrs=127.8.241.254:43515,127.8.241.253:41923,127.8.241.252:33933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:11.904564 19770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:11.905632 19770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:11.915165 19920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.915174 19919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:11.915174 19924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:11.915489 19770 server_base.cc:1034] running on GCE node
I20260806 18:40:11.915920 19770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:11.916584 19770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:11.917744 19770 hybrid_clock.cc:648] HybridClock initialized: now 1786041611917720 us; error 39 us; skew 500 ppm
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:11.920459 19770 init.cc:377] Logged in from keytab as oryx/127.8.241.193@KRBTEST.COM (short username oryx)
I20260806 18:40:11.921366 19770 webserver.cc:460] Webserver started at http://127.8.241.193:35227/ using document root <none> and password file <none>
I20260806 18:40:11.921955 19770 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:11.922070 19770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:11.926141 19770 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260806 18:40:11.928210 19938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.929464 19770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:40:11.929580 19770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "c6ff04d70057445c801791a432a44a1f"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "b67951fef0b68c9acc7f5fb6ae95f5ea"
server_key_iv: "2ec924b092e13246499a72eacd678cca"
server_key_version: "encryptionkey@0"
I20260806 18:40:11.930167 19770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:11.955070 19770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:11.958381 19770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:11.958587 19770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:11.959203 19770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:11.960130 19770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:11.960182 19770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.960238 19770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:11.960264 19770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:11.969131 19770 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:44305
I20260806 18:40:11.969161 20112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:44305 every 8 connection(s)
I20260806 18:40:11.970296 19770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:11.980540 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 19770
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.253@KRBTEST.COM
I20260806 18:40:11.980731 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 18862
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:11.986603 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.973684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:41629 (local address 127.8.241.254:43515)
0806 18:40:11.973812 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.973816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.973828 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:40:11.974934 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.974936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.974996 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.975054 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.976336 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.977150 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.978345 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.978679 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.978768 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.982084 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.982088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.982090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.982119 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.983691 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.984439 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.984444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.984446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.984531 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.984983 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.984987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.984990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.985298 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.985386 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.986292 (+ 906us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.986431 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:40:11.986624 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.972146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:52135 (local address 127.8.241.253:41923)
0806 18:40:11.972298 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.972302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.973728 (+ 1426us) server_negotiation.cc:407] Connection header received
0806 18:40:11.974681 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.974685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.974747 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.974818 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.976325 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.976839 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.978387 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.978620 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.978709 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.983616 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.983619 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.983621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.983645 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:11.985104 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.985487 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.985490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.985491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.985565 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:11.985819 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:11.985822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:11.985824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:11.986006 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:11.986090 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:11.986376 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 18:40:11.986499 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:40:11.987887 20124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.972578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.8.241.193:52135)
0806 18:40:11.973633 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.973645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.974460 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.974996 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.975003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.975579 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.976174 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.976184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.976992 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.976994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.978242 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.978253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.978763 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.978767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.978927 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.979485 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.979508 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.983478 (+ 3970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.985199 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.985203 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.985205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.985387 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.985646 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.985649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.985651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.985730 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.986097 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.986101 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.986192 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.986828 (+ 636us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.986975 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"mutex_wait_us":196,"spinlock_wait_cycles":55424,"thread_start_us":140,"threads_started":1}
I20260806 18:40:11.988205 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.972578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:41629)
0806 18:40:11.973424 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.973490 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.974460 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.975095 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.975098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.975579 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.976177 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.976184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.977258 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.977262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.978242 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.978253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.978912 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.978916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.979040 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.979759 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.979772 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.981877 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:11.983845 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.983853 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.983875 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.984289 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.984630 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:11.984633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:11.984636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:11.984855 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:11.985499 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:11.985507 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:11.985946 (+ 439us) client_negotiation.cc:769] Sending connection context
0806 18:40:11.986229 (+ 283us) client_negotiation.cc:240] Negotiation successful
0806 18:40:11.986917 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":82,"spinlock_wait_cycles":34432,"thread_start_us":146,"threads_started":1}
I20260806 18:40:11.989487 20118 heartbeater.cc:347] Connected to a master server at 127.8.241.253:41923
I20260806 18:40:11.989749 20118 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:11.990322 20118 heartbeater.cc:511] Master 127.8.241.253:41923 requested a full tablet report, sending...
I20260806 18:40:11.992148 19528 ts_manager.cc:194] Registered new tserver with Master: c6ff04d70057445c801791a432a44a1f (127.8.241.193:44305)
I20260806 18:40:11.998222 20117 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43515
Aug 06 18:40:11 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041611, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.252@KRBTEST.COM
I20260806 18:40:11.998330 20117 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:11.998551 20117 heartbeater.cc:511] Master 127.8.241.254:43515 requested a full tablet report, sending...
I20260806 18:40:11.999485 18507 ts_manager.cc:194] Re-registered known tserver with Master: c6ff04d70057445c801791a432a44a1f (127.8.241.193:44305)
I20260806 18:40:12.000272 18507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:41629
I20260806 18:40:12.002240 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.972908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:56291 (local address 127.8.241.252:33933)
0806 18:40:11.973173 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:11.973186 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:11.974119 (+ 933us) server_negotiation.cc:407] Connection header received
0806 18:40:11.975159 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:11.975162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:11.975204 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:11.975277 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:11.976310 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.977383 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.978309 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.978669 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.978768 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.998951 (+ 20183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:11.998955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:11.998959 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:11.998989 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.000547 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.001064 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.001068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.001070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.001159 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.001416 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.001420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.001421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.001612 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.001714 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.001966 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.002093 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":91,"threads_started":1}
I20260806 18:40:12.002254 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:11.972769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.8.241.193:56291)
0806 18:40:11.974032 (+ 1263us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:11.974045 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:11.974471 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:11.975499 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:11.975501 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:11.975668 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:11.976162 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.976177 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.977490 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.977493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.978192 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:11.978201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:11.978895 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:11.978899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:11.979030 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:11.979678 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:11.979689 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:11.998744 (+ 19055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:12.000679 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.000682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.000684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.000941 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.001245 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.001249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.001251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.001335 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.001703 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:12.001707 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:12.001803 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.001892 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.002057 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1204,"mutex_wait_us":36,"spinlock_wait_cycles":8576,"thread_start_us":113,"threads_started":1}
I20260806 18:40:12.003216 20113 heartbeater.cc:347] Connected to a master server at 127.8.241.252:33933
I20260806 18:40:12.003311 20113 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:12.003522 20113 heartbeater.cc:511] Master 127.8.241.252:33933 requested a full tablet report, sending...
I20260806 18:40:12.004762 19833 ts_manager.cc:194] Registered new tserver with Master: c6ff04d70057445c801791a432a44a1f (127.8.241.193:44305)
I20260806 18:40:12.005203 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:39639
--local_ip_for_outbound_sockets=127.8.241.194
--tserver_master_addrs=127.8.241.254:43515
--webserver_port=36033
--webserver_interface=127.8.241.194
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab
--principal=oryx/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.241.254:43515,127.8.241.253:41923
--tserver_master_addrs=127.8.241.254:43515,127.8.241.253:41923,127.8.241.252:33933 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:12.078141 19271 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.8.241.195:37419
I20260806 18:40:12.095189 19278 heartbeater.cc:503] Master 127.8.241.254:43515 was elected leader, sending a full tablet report...
W20260806 18:40:12.121630 20128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:12.121891 20128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:12.121959 20128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:12.125873 20128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:12.125931 20128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:12.126011 20128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:12.131141 20128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab
--principal=oryx/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:39639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=36033
--tserver_master_addrs=127.8.241.254:43515,127.8.241.253:41923,127.8.241.252:33933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:12.132222 20128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:12.133319 20128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:12.143246 20135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:12.143262 20139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:12.143392 20128 server_base.cc:1034] running on GCE node
W20260806 18:40:12.143292 20141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:12.143967 20128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:12.144598 20128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:12.145752 20128 hybrid_clock.cc:648] HybridClock initialized: now 1786041612145720 us; error 44 us; skew 500 ppm
Aug 06 18:40:12 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041612, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:12.148509 20128 init.cc:377] Logged in from keytab as oryx/127.8.241.194@KRBTEST.COM (short username oryx)
I20260806 18:40:12.149439 20128 webserver.cc:460] Webserver started at http://127.8.241.194:36033/ using document root <none> and password file <none>
I20260806 18:40:12.150004 20128 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:12.150149 20128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:12.154124 20128 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 18:40:12.156189 20149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:12.157444 20128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:40:12.157575 20128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "87ac15e9acfb46c08395ce0aa8819ccc"
format_stamp: "Formatted at 2026-08-06 18:40:10 on dist-test-slave-k5rr"
server_key: "e3ae06699caf549e617b4c6c262e5734"
server_key_iv: "c9e340bef3608f61447ef721d90da6ca"
server_key_version: "encryptionkey@0"
I20260806 18:40:12.158133 20128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:12.172745 20128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:12.175683 20128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:12.175866 20128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:12.176496 20128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:12.177585 20128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:12.177663 20128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:12.177727 20128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:12.177768 20128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:12.187235 20128 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:39639
I20260806 18:40:12.187242 20322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:39639 every 8 connection(s)
I20260806 18:40:12.188328 20128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:12.191422 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 20128
I20260806 18:40:12.191617 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 19070
Aug 06 18:40:12 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041612, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:12.206637 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.190272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:59323 (local address 127.8.241.254:43515)
0806 18:40:12.190401 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.190406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.192047 (+ 1641us) server_negotiation.cc:407] Connection header received
0806 18:40:12.192864 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.192867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.192924 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.192976 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.197556 (+ 4580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.198181 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.200089 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.200358 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.200429 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.202900 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.202904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.202906 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.202931 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.204262 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.204807 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.204810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.204812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.204885 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.205240 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.205243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.205245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.205441 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.205512 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.206330 (+ 818us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.206463 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 18:40:12 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041612, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.252@KRBTEST.COM
I20260806 18:40:12.208776 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.190683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.194:59323)
0806 18:40:12.191962 (+ 1279us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.191974 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.192767 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.193341 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.193347 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.193693 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:12.194348 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.194376 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.198304 (+ 3928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.198306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.198977 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.198985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.200588 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.200591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.200675 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.201139 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:12.201158 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:12.202726 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:12.204387 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.204392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.204408 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.204710 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.204983 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.204985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.204987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.205154 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.205591 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:12.205597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:12.205977 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.206920 (+ 943us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.207683 (+ 763us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1187,"mutex_wait_us":1183,"thread_start_us":108,"threads_started":1}
I20260806 18:40:12.210496 20325 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43515
I20260806 18:40:12.210760 20325 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:12.211014 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.190776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:42887 (local address 127.8.241.252:33933)
0806 18:40:12.190904 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.190907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.191823 (+ 916us) server_negotiation.cc:407] Connection header received
0806 18:40:12.194574 (+ 2751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.194579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.194639 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.194699 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.195405 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.196157 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.199093 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.199364 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.199423 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.207490 (+ 8067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.207495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.207497 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.207527 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.209180 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.209685 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.209689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.209692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.209775 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.210059 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.210063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.210065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.210267 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.210347 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.210687 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.210828 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:40:12.211370 20325 heartbeater.cc:511] Master 127.8.241.254:43515 requested a full tablet report, sending...
I20260806 18:40:12.211578 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.190683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.8.241.194:42887)
0806 18:40:12.191732 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.191746 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.192603 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.194864 (+ 2261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.194867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.195100 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:12.195247 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.195256 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.196312 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.196315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.197427 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.197437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.199600 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.199603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.199780 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.205775 (+ 5995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:12.205788 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:12.207312 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:12.209318 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.209321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.209323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.209570 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.209872 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.209875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.209876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.209965 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.210404 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:12.210407 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:12.210505 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.211187 (+ 682us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.211320 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":26,"spinlock_wait_cycles":33536,"thread_start_us":121,"threads_started":1}
I20260806 18:40:12.212356 20324 heartbeater.cc:347] Connected to a master server at 127.8.241.252:33933
I20260806 18:40:12.212461 20324 heartbeater.cc:464] Registering TS with master...
Aug 06 18:40:12 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041612, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.253@KRBTEST.COM
I20260806 18:40:12.212641 20324 heartbeater.cc:511] Master 127.8.241.252:33933 requested a full tablet report, sending...
I20260806 18:40:12.212831 18507 ts_manager.cc:194] Re-registered known tserver with Master: 87ac15e9acfb46c08395ce0aa8819ccc (127.8.241.194:39639)
I20260806 18:40:12.213444 19833 ts_manager.cc:194] Registered new tserver with Master: 87ac15e9acfb46c08395ce0aa8819ccc (127.8.241.194:39639)
I20260806 18:40:12.213608 18507 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:59323
I20260806 18:40:12.214109 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:37419
--local_ip_for_outbound_sockets=127.8.241.195
--tserver_master_addrs=127.8.241.254:43515
--webserver_port=43351
--webserver_interface=127.8.241.195
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab
--principal=oryx/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.241.254:43515,127.8.241.253:41923
--tserver_master_addrs=127.8.241.254:43515,127.8.241.253:41923,127.8.241.252:33933 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:40:12.216526 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.190244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:34461 (local address 127.8.241.253:41923)
0806 18:40:12.190411 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.190417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.191379 (+ 962us) server_negotiation.cc:407] Connection header received
0806 18:40:12.206234 (+ 14855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.206238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.206299 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.206355 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.208091 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.208874 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.209994 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.210301 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.210373 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.213116 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.213119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.213122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.213150 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.214918 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.215427 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.215431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.215433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.215526 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.215825 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.215827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.215828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.215998 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.216068 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.216327 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.216416 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 18:40:12.216620 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.190683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.8.241.194:34461)
0806 18:40:12.191216 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.191275 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.206070 (+ 14795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.206513 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.206516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.207057 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:12.207352 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.207363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.209063 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.209067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.209859 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.209868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.210540 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.210543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.210632 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.211113 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:12.211126 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:12.212939 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:12.215052 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.215055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.215057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.215314 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.215615 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.215618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.215620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.215736 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.216078 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:12.216081 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:12.216178 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.216284 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.216435 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":3,"thread_start_us":149,"threads_started":1}
I20260806 18:40:12.217229 20327 heartbeater.cc:347] Connected to a master server at 127.8.241.253:41923
I20260806 18:40:12.217319 20327 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:12.217505 20327 heartbeater.cc:511] Master 127.8.241.253:41923 requested a full tablet report, sending...
I20260806 18:40:12.218240 19528 ts_manager.cc:194] Registered new tserver with Master: 87ac15e9acfb46c08395ce0aa8819ccc (127.8.241.194:39639)
I20260806 18:40:12.282475 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.279108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.8.241.252:33933)
0806 18:40:12.279244 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.279248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.279341 (+ 93us) server_negotiation.cc:407] Connection header received
0806 18:40:12.279504 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.279507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.279562 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.279644 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:12.279899 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.280416 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.281503 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.281926 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.282023 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.282179 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.282262 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.282327 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:40:12.282476 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.279019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.0.0.1:59888)
0806 18:40:12.279244 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.279259 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.279390 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.279666 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.279670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.279692 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:12.279792 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.279798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.280535 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.280539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.281360 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.281371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.282044 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.282048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.282140 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.282165 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.282248 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.282327 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":63,"threads_started":1}
I20260806 18:40:12.283353 19848 raft_consensus.cc:1203] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 18:40:12.283700 19893 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bc7173999834375bfaeaf8f77292c24. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:12.283854 19893 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: This master's current role is: LEARNER
W20260806 18:40:12.326855 20336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:12.327095 20336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:12.327150 20336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:12.331024 20336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:12.331079 20336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:12.331152 20336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:12.336243 20336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab
--principal=oryx/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:37419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=43351
--tserver_master_addrs=127.8.241.254:43515,127.8.241.253:41923,127.8.241.252:33933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:12.337257 20336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:12.338230 20336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:12.347903 20351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:12.347891 20349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:12.348225 20336 server_base.cc:1034] running on GCE node
W20260806 18:40:12.348412 20353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:12.348811 20336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:12.349557 20336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:12.350708 20336 hybrid_clock.cc:648] HybridClock initialized: now 1786041612350672 us; error 46 us; skew 500 ppm
Aug 06 18:40:12 dist-test-slave-k5rr krb5kdc[18438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041612, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:12.353629 20336 init.cc:377] Logged in from keytab as oryx/127.8.241.195@KRBTEST.COM (short username oryx)
I20260806 18:40:12.354594 20336 webserver.cc:460] Webserver started at http://127.8.241.195:43351/ using document root <none> and password file <none>
I20260806 18:40:12.355229 20336 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:12.355312 20336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:12.359762 20336 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:40:12.361826 20365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:12.363035 20336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:40:12.363144 20336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b"
format_stamp: "Formatted at 2026-08-06 18:40:11 on dist-test-slave-k5rr"
server_key: "839cf65a241885f3670790cc3f64354f"
server_key_iv: "bdea70d1db5b2cdcf08fe27500bd34f6"
server_key_version: "encryptionkey@0"
I20260806 18:40:12.363664 20336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:12.393224 20336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:12.396476 20336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:12.396636 20336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:12.397230 20336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:12.398089 20336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:12.398136 20336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:12.398175 20336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:12.398195 20336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:12.406646 20336 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:37419
I20260806 18:40:12.406703 20536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:37419 every 8 connection(s)
I20260806 18:40:12.407670 20336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:12.411268 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 20336
I20260806 18:40:12.423044 20556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.413096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41836 (local address 127.8.241.253:41923)
0806 18:40:12.414168 (+ 1072us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.414172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.414185 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:12.414227 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.414229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.414273 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.414361 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.415117 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.415861 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.416913 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.417255 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.417336 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.419129 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.419133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.419135 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.419157 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.420856 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.421321 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.421325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.421327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.421416 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.421748 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.421751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.421753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.421979 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.422106 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.422684 (+ 578us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.422865 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":999,"thread_start_us":60,"threads_started":1}
Aug 06 18:40:12 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041612, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
Aug 06 18:40:12 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041612, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.253@KRBTEST.COM
I20260806 18:40:12.425844 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.413112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.8.241.254:43515)
0806 18:40:12.413736 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.413749 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.413763 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:12.413806 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.413809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.413855 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.413955 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.414783 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.415636 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.416817 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.417207 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.417304 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.419657 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.419661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.419663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.419686 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.423582 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.424118 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.424122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.424124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.424211 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.424537 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.424540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.424542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.424748 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.424865 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.425561 (+ 696us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.425681 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":73,"threads_started":1}
I20260806 18:40:12.426085 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.413137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.8.241.252:33933)
0806 18:40:12.414584 (+ 1447us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.414588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.414883 (+ 295us) server_negotiation.cc:407] Connection header received
0806 18:40:12.415053 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.415056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.415106 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.415165 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.416087 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.416864 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.417990 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.418343 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.418453 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.420565 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.420569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.420572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.420600 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.422379 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.424118 (+ 1739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.424122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.424124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.424211 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.424550 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.424553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.424554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.424748 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.424865 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.425808 (+ 943us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.425929 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1372,"thread_start_us":81,"threads_started":1}
I20260806 18:40:12.430048 18583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.409462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:42299 (local address 127.8.241.254:43515)
0806 18:40:12.409595 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.409599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.410662 (+ 1063us) server_negotiation.cc:407] Connection header received
0806 18:40:12.414598 (+ 3936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.414602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.414650 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.414722 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.416403 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.417143 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.419067 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.419374 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.419477 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.424994 (+ 5517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.424997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.425000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.425053 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.427682 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.428287 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.428290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.428292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.428383 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.428821 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.428824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.428826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.429055 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.429142 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.429798 (+ 656us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.429905 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 18:40:12 dist-test-slave-k5rr krb5kdc[18438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041612, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.252@KRBTEST.COM
I20260806 18:40:12.430768 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.409777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:55517 (local address 127.8.241.253:41923)
0806 18:40:12.409885 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.409890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.411162 (+ 1272us) server_negotiation.cc:407] Connection header received
0806 18:40:12.419115 (+ 7953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.419118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.419163 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.419209 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.419824 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.420474 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.421603 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.421912 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.421999 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.424743 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.424746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.424749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.424775 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.428900 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.429368 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.429371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.429373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.429486 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.429789 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.429792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.429794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.430006 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.430096 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.430493 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.430626 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:40:12.428754 18507 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:40:12.431680 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.409982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.195:42299)
0806 18:40:12.410496 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.410551 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.413614 (+ 3063us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x1b394de8. stack: 00007f03b4179c78 00007f03b417a1a9 00007f03b3d3c1d7 00000000005984a6 00000000005984f9 000000000059c488 00007f03b41762ab 00007f03b4175ad0 00007f03bcf4d0a5 00007f03bcf4d58c 00007f03bc8dc043 00007f03b87ce267 00007f03b87cc819 00007f03b880d832 00007f03b880e68c 00007f03b88240d3
0806 18:40:12.414279 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.415044 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.415054 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.415554 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:12.416244 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.416263 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.417595 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.417599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.418939 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.418964 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.419583 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.419586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.421292 (+ 1706us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.421940 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:12.421960 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:12.424835 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:12.427791 (+ 2956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.427797 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.427816 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.428167 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.428522 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.428525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.428527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.428710 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.429196 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:12.429203 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:12.429523 (+ 320us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.430189 (+ 666us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.430570 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":1462,"spinlock_wait_cycles":4338944,"thread_start_us":125,"threads_started":1}
I20260806 18:40:12.431677 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.409982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.8.241.195:55517)
0806 18:40:12.411062 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.411075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.417407 (+ 6332us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x1b394de8. stack: 00007f03b4179c78 00007f03b417a1a9 00007f03b3d3c1d7 00000000005984a6 00000000005984f9 000000000059c488 00007f03b41762ab 00007f03b41759da 00007f03bc8d3817 00007f03bc8e607f 00007f03bc8dc003 00007f03b87ce267 00007f03b87cc819 00007f03b880d832 00007f03b880e68c 00007f03b88240d3
0806 18:40:12.417431 (+ 24us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.419354 (+ 1923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.419357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.419528 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:12.419688 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.419697 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.420628 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.420632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.421468 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.421479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.422131 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.422134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.422222 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.422757 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:12.422771 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:12.424458 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:12.428988 (+ 4530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.428991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.428993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.429250 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.429594 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.429597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.429599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.429682 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.430142 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:12.430146 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:12.430242 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.430333 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.431076 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"mutex_wait_us":694,"spinlock_wait_cycles":3916032,"thread_start_us":148,"threads_started":1}
W20260806 18:40:12.432797 18507 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:12.432979 20545 heartbeater.cc:347] Connected to a master server at 127.8.241.253:41923
I20260806 18:40:12.433269 20545 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:12.433876 20545 heartbeater.cc:511] Master 127.8.241.253:41923 requested a full tablet report, sending...
I20260806 18:40:12.435091 20543 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43515
I20260806 18:40:12.435178 20543 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:12.435242 19528 ts_manager.cc:194] Registered new tserver with Master: 79bd32d76dd24d43b2bc5aacc4e72f3b (127.8.241.195:37419)
I20260806 18:40:12.435346 20543 heartbeater.cc:511] Master 127.8.241.254:43515 requested a full tablet report, sending...
I20260806 18:40:12.435566 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.409701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:50285 (local address 127.8.241.252:33933)
0806 18:40:12.409827 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.409831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.410799 (+ 968us) server_negotiation.cc:407] Connection header received
0806 18:40:12.421683 (+ 10884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.421686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.421737 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.421782 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.425516 (+ 3734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.426234 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.427286 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.427606 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.427718 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.431295 (+ 3577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.431299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.431301 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.431325 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.433569 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.434159 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.434163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.434164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.434251 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.434595 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.434599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.434600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.434898 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.434998 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.435289 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.435408 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:40:12.436218 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.409982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.8.241.195:50285)
0806 18:40:12.410701 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.410715 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.421552 (+ 10837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.425054 (+ 3502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.425056 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.425239 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:12.425362 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.425370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.426374 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.426378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.427165 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.427177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.428194 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.428198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.428287 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.429350 (+ 1063us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:12.429363 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:12.431136 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:12.433683 (+ 2547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.433686 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.433688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.434033 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.434349 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:12.434352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:12.434353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:12.434480 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:12.434991 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:12.434994 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:12.435091 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.435701 (+ 610us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.435843 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":22,"spinlock_wait_cycles":7040,"thread_start_us":188,"threads_started":1}
I20260806 18:40:12.436265 18505 ts_manager.cc:194] Re-registered known tserver with Master: 79bd32d76dd24d43b2bc5aacc4e72f3b (127.8.241.195:37419)
I20260806 18:40:12.436874 20539 heartbeater.cc:347] Connected to a master server at 127.8.241.252:33933
I20260806 18:40:12.436825 20337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: NON_VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: true } } }
I20260806 18:40:12.436939 20539 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:12.436923 18505 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:42299
I20260806 18:40:12.436964 20337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:12.437127 20539 heartbeater.cc:511] Master 127.8.241.252:33933 requested a full tablet report, sending...
I20260806 18:40:12.439908 19833 ts_manager.cc:194] Registered new tserver with Master: 79bd32d76dd24d43b2bc5aacc4e72f3b (127.8.241.195:37419)
I20260806 18:40:12.440796 20337 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: attempting to promote NON_VOTER eb579dc4a65a4555b6c1011b3a874f74 to VOTER
I20260806 18:40:12.441309 20337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } }
I20260806 18:40:12.442646 19848 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 LEARNER]: Refusing update from remote peer 7bc7173999834375bfaeaf8f77292c24: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:40:12.443465 20566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 18:40:12.444236 19535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Refusing update from remote peer 7bc7173999834375bfaeaf8f77292c24: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:40:12.444661 20566 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 18:40:12.446218 19848 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:12.447026 19535 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:12.447045 20337 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:12.448469 19893 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:12.448581 19893 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:40:12.449090 20582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:12.449218 20582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:40:12.449307 20566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:12.449404 20566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:12.449592 20337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:12.449702 20337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:12.449790 20566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7bc7173999834375bfaeaf8f77292c24" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:12.449873 20566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:12.456640 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.451443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37419 (local address 127.0.0.1:37422)
0806 18:40:12.451566 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.451581 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.451661 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.453237 (+ 1576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.453240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.453259 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:12.453345 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.453351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.454443 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.454446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.455178 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.455187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.456253 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.456257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.456345 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.456375 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.456418 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.456503 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:40:12.457305 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.451841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.8.241.195:37419)
0806 18:40:12.452727 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.452740 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.452765 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:40:12.452844 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.452849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.453107 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.453263 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:12.453437 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.454339 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.455298 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.456118 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.456831 (+ 713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.456981 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.457069 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.457165 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":133,"threads_started":1}
I20260806 18:40:12.458020 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.452212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:39639 (local address 127.0.0.1:35640)
0806 18:40:12.452425 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.452437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.452508 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.453432 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.453435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.453453 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:12.453539 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.453545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.455440 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.455443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.456559 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.456568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.457617 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.457621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.457708 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.457730 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.457815 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.457889 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":29,"thread_start_us":96,"threads_started":1}
I20260806 18:40:12.458068 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.452616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.8.241.194:39639)
0806 18:40:12.452941 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.452953 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.452974 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:40:12.453071 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.453076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.453281 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.453684 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:12.453827 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.455335 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.456679 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.457508 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.457609 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.457746 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.457842 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.457941 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":99,"threads_started":1}
I20260806 18:40:12.458346 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.451890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44305 (local address 127.0.0.1:33628)
0806 18:40:12.452124 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.452137 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.452223 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.452898 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.452901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.452920 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:12.453071 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.453078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.455478 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.455482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.456286 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.456294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.457984 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.457988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.458076 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.458103 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.458153 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.458221 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":20,"thread_start_us":110,"threads_started":1}
I20260806 18:40:12.458937 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.452084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.8.241.193:44305)
0806 18:40:12.452368 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.452383 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.452408 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:40:12.452498 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.452504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.452725 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.454026 (+ 1301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:12.454185 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.455335 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.456427 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.457831 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.458525 (+ 694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.458667 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.458716 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.458800 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":92,"threads_started":1}
I20260806 18:40:12.460569 20441 tablet_service.cc:1467] Processing CreateTablet for tablet afd08ed0f3184b5aa9a6640b4dc77407 (DEFAULT_TABLE table=test-table [id=d24697dde4ac4a0cae9a85bab7c04b39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:12.460755 20226 tablet_service.cc:1467] Processing CreateTablet for tablet afd08ed0f3184b5aa9a6640b4dc77407 (DEFAULT_TABLE table=test-table [id=d24697dde4ac4a0cae9a85bab7c04b39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:12.461712 20441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd08ed0f3184b5aa9a6640b4dc77407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:12.461870 20226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd08ed0f3184b5aa9a6640b4dc77407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:12.461613 20006 tablet_service.cc:1467] Processing CreateTablet for tablet afd08ed0f3184b5aa9a6640b4dc77407 (DEFAULT_TABLE table=test-table [id=d24697dde4ac4a0cae9a85bab7c04b39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:12.462555 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afd08ed0f3184b5aa9a6640b4dc77407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:12.468798 20592 tablet_bootstrap.cc:492] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: Bootstrap starting.
I20260806 18:40:12.471349 20592 tablet_bootstrap.cc:654] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:12.472074 20592 log.cc:824] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:12.472085 20594 tablet_bootstrap.cc:492] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: Bootstrap starting.
I20260806 18:40:12.472636 20593 tablet_bootstrap.cc:492] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: Bootstrap starting.
I20260806 18:40:12.474308 20594 tablet_bootstrap.cc:654] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:12.474488 20592 tablet_bootstrap.cc:492] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: No bootstrap required, opened a new log
I20260806 18:40:12.474677 20592 ts_tablet_manager.cc:1397] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:40:12.474799 20593 tablet_bootstrap.cc:654] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:12.475179 20594 log.cc:824] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:12.475591 20593 log.cc:824] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:12.477357 20594 tablet_bootstrap.cc:492] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: No bootstrap required, opened a new log
I20260806 18:40:12.477444 20593 tablet_bootstrap.cc:492] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: No bootstrap required, opened a new log
I20260806 18:40:12.477567 20594 ts_tablet_manager.cc:1397] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260806 18:40:12.477630 20593 ts_tablet_manager.cc:1397] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:40:12.479143 20592 raft_consensus.cc:348] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } }
I20260806 18:40:12.479444 20592 raft_consensus.cc:374] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:12.479516 20592 raft_consensus.cc:729] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bd32d76dd24d43b2bc5aacc4e72f3b, State: Initialized, Role: FOLLOWER
I20260806 18:40:12.480016 20592 consensus_queue.cc:260] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } }
I20260806 18:40:12.480760 20543 heartbeater.cc:503] Master 127.8.241.254:43515 was elected leader, sending a full tablet report...
I20260806 18:40:12.481168 20592 ts_tablet_manager.cc:1428] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260806 18:40:12.484397 20594 raft_consensus.cc:348] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } }
I20260806 18:40:12.484712 20594 raft_consensus.cc:374] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:12.484776 20594 raft_consensus.cc:729] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6ff04d70057445c801791a432a44a1f, State: Initialized, Role: FOLLOWER
I20260806 18:40:12.485297 20594 consensus_queue.cc:260] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } }
I20260806 18:40:12.485509 20593 raft_consensus.cc:348] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } }
I20260806 18:40:12.485718 20593 raft_consensus.cc:374] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:12.485774 20593 raft_consensus.cc:729] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ac15e9acfb46c08395ce0aa8819ccc, State: Initialized, Role: FOLLOWER
I20260806 18:40:12.487071 20594 ts_tablet_manager.cc:1428] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: Time spent starting tablet: real 0.009s user 0.002s sys 0.005s
I20260806 18:40:12.487247 20117 heartbeater.cc:503] Master 127.8.241.254:43515 was elected leader, sending a full tablet report...
I20260806 18:40:12.489274 20593 consensus_queue.cc:260] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } }
I20260806 18:40:12.489893 20325 heartbeater.cc:503] Master 127.8.241.254:43515 was elected leader, sending a full tablet report...
I20260806 18:40:12.490245 20593 ts_tablet_manager.cc:1428] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: Time spent starting tablet: real 0.013s user 0.003s sys 0.002s
I20260806 18:40:12.606700 20602 raft_consensus.cc:482] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:12.606977 20602 raft_consensus.cc:504] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } }
I20260806 18:40:12.608110 20602 leader_election.cc:288] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6ff04d70057445c801791a432a44a1f (127.8.241.193:44305), 79bd32d76dd24d43b2bc5aacc4e72f3b (127.8.241.195:37419)
I20260806 18:40:12.611928 20588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.608523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:46517 (local address 127.8.241.193:44305)
0806 18:40:12.608755 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.608760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.608775 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:40:12.608830 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.608833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.608884 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.608956 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:12.609257 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.609740 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.610876 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.611308 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.611416 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.611486 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.611720 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.611795 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260806 18:40:12.611932 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.608343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44305 (local address 127.8.241.194:46517)
0806 18:40:12.608501 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.608515 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.608613 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.608976 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.608978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.608995 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:12.609148 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.609154 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.609848 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.609851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.610752 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.610762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.611416 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.611420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.611600 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.611623 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.611691 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.611793 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":125,"spinlock_wait_cycles":2304}
I20260806 18:40:12.611933 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.608484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:60039 (local address 127.8.241.195:37419)
0806 18:40:12.608625 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.608630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.608737 (+ 107us) server_negotiation.cc:407] Connection header received
0806 18:40:12.608850 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.608853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.608893 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.608962 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:12.609249 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.609740 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.610889 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.611316 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.611422 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.611490 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.611734 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.611807 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:40:12.611958 20332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.608392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37419 (local address 127.8.241.194:60039)
0806 18:40:12.608659 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:12.608672 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:12.608772 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:12.608993 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:12.608996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:12.609035 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:12.609146 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.609151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.609883 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.609886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.610751 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:12.610760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.611423 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.611426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:12.611593 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.611635 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 18:40:12.611703 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:40:12.611795 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":175}
I20260806 18:40:12.612864 20037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd08ed0f3184b5aa9a6640b4dc77407" candidate_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6ff04d70057445c801791a432a44a1f" is_pre_election: true
I20260806 18:40:12.613225 20037 raft_consensus.cc:2454] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87ac15e9acfb46c08395ce0aa8819ccc in term 0.
I20260806 18:40:12.613483 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd08ed0f3184b5aa9a6640b4dc77407" candidate_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" is_pre_election: true
I20260806 18:40:12.613793 20472 raft_consensus.cc:2454] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87ac15e9acfb46c08395ce0aa8819ccc in term 0.
I20260806 18:40:12.613757 20151 leader_election.cc:302] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87ac15e9acfb46c08395ce0aa8819ccc, c6ff04d70057445c801791a432a44a1f; no voters:
I20260806 18:40:12.614078 20602 raft_consensus.cc:2781] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:12.614189 20602 raft_consensus.cc:482] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:12.614248 20602 raft_consensus.cc:3037] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:12.615298 20602 raft_consensus.cc:504] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } }
I20260806 18:40:12.615649 20602 leader_election.cc:288] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [CANDIDATE]: Term 1 election: Requested vote from peers c6ff04d70057445c801791a432a44a1f (127.8.241.193:44305), 79bd32d76dd24d43b2bc5aacc4e72f3b (127.8.241.195:37419)
I20260806 18:40:12.616004 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd08ed0f3184b5aa9a6640b4dc77407" candidate_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b"
I20260806 18:40:12.616004 20037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afd08ed0f3184b5aa9a6640b4dc77407" candidate_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6ff04d70057445c801791a432a44a1f"
I20260806 18:40:12.616137 20472 raft_consensus.cc:3037] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:12.616137 20037 raft_consensus.cc:3037] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:12.617380 20037 raft_consensus.cc:2454] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87ac15e9acfb46c08395ce0aa8819ccc in term 1.
I20260806 18:40:12.617380 20472 raft_consensus.cc:2454] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87ac15e9acfb46c08395ce0aa8819ccc in term 1.
I20260806 18:40:12.617722 20156 leader_election.cc:302] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79bd32d76dd24d43b2bc5aacc4e72f3b, 87ac15e9acfb46c08395ce0aa8819ccc; no voters:
I20260806 18:40:12.617931 20602 raft_consensus.cc:2781] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:12.618155 20602 raft_consensus.cc:686] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 1 LEADER]: Becoming Leader. State: Replica: 87ac15e9acfb46c08395ce0aa8819ccc, State: Running, Role: LEADER
I20260806 18:40:12.618526 20602 consensus_queue.cc:237] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } }
I20260806 18:40:12.621881 18507 catalog_manager.cc:5495] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc reported cstate change: term changed from 0 to 1, leader changed from <none> to 87ac15e9acfb46c08395ce0aa8819ccc (127.8.241.194). New cstate: current_term: 1 leader_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } health_report { overall_health: HEALTHY } } }
I20260806 18:40:12.648018 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.644986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.8.241.194:39639)
0806 18:40:12.645135 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.645139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.645153 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:12.645233 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.645235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.645290 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.645377 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:12.645560 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.646055 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.646797 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.647000 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.647066 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.647284 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:12.647719 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:12.647806 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.647831 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.647879 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:40:12.654129 20472 raft_consensus.cc:1261] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 1 FOLLOWER]: Refusing update from remote peer 87ac15e9acfb46c08395ce0aa8819ccc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:12.654129 20037 raft_consensus.cc:1261] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [term 1 FOLLOWER]: Refusing update from remote peer 87ac15e9acfb46c08395ce0aa8819ccc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:12.654873 20603 consensus_queue.cc:1035] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [LEADER]: Connected to new peer: Peer: permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:12.655056 20602 consensus_queue.cc:1035] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:40:12.659442 20546 tablet.cc:2369] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:40:12.665791 20615 mvcc.cc:205] Tried to move back new op lower bound from 7315626445424078848 to 7315626445287985152. Current Snapshot: MvccSnapshot[applied={T|T < 7315626445424078848}]
I20260806 18:40:12.669770 20613 mvcc.cc:205] Tried to move back new op lower bound from 7315626445424078848 to 7315626445287985152. Current Snapshot: MvccSnapshot[applied={T|T < 7315626445424078848 or (T in {7315626445424078848})}]
I20260806 18:40:12.675871 18507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:12.676020 18507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:12.680248 18507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7bc7173999834375bfaeaf8f77292c24: Sending DeleteTablet for 3 replicas of tablet afd08ed0f3184b5aa9a6640b4dc77407
I20260806 18:40:12.681031 20006 tablet_service.cc:1514] Processing DeleteTablet for tablet afd08ed0f3184b5aa9a6640b4dc77407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:12 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:33628
I20260806 18:40:12.681578 20631 tablet_replica.cc:330] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: stopping tablet replica
I20260806 18:40:12.681516 20226 tablet_service.cc:1514] Processing DeleteTablet for tablet afd08ed0f3184b5aa9a6640b4dc77407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:12 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:35640
I20260806 18:40:12.681541 20441 tablet_service.cc:1514] Processing DeleteTablet for tablet afd08ed0f3184b5aa9a6640b4dc77407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:12 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:37422
I20260806 18:40:12.681927 20631 raft_consensus.cc:2229] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:12.682219 20637 tablet_replica.cc:330] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: stopping tablet replica
I20260806 18:40:12.682300 20631 raft_consensus.cc:2258] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:12.682610 20637 raft_consensus.cc:2229] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:40:12.683068 20637 raft_consensus.cc:2258] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:12.683547 20634 tablet_replica.cc:330] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: stopping tablet replica
I20260806 18:40:12.684211 20631 ts_tablet_manager.cc:1905] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:12.684574 20634 raft_consensus.cc:2229] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:12.684613 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 18459
I20260806 18:40:12.690768 20637 ts_tablet_manager.cc:1905] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:12.691207 20634 raft_consensus.cc:2258] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:12.694156 20637 ts_tablet_manager.cc:1918] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:12.694257 20631 ts_tablet_manager.cc:1918] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:12.694373 20631 log.cc:1196] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/afd08ed0f3184b5aa9a6640b4dc77407
I20260806 18:40:12.694525 20637 log.cc:1196] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/afd08ed0f3184b5aa9a6640b4dc77407
I20260806 18:40:12.694732 20631 ts_tablet_manager.cc:1939] T afd08ed0f3184b5aa9a6640b4dc77407 P c6ff04d70057445c801791a432a44a1f: Deleting consensus metadata
I20260806 18:40:12.695008 20637 ts_tablet_manager.cc:1939] T afd08ed0f3184b5aa9a6640b4dc77407 P 87ac15e9acfb46c08395ce0aa8819ccc: Deleting consensus metadata
I20260806 18:40:12.695822 20634 ts_tablet_manager.cc:1905] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:12.698361 20634 ts_tablet_manager.cc:1918] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:12.698467 20634 log.cc:1196] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/afd08ed0f3184b5aa9a6640b4dc77407
I20260806 18:40:12.698769 20634 ts_tablet_manager.cc:1939] T afd08ed0f3184b5aa9a6640b4dc77407 P 79bd32d76dd24d43b2bc5aacc4e72f3b: Deleting consensus metadata
I20260806 18:40:12.705281 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.683170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.8.241.253:41923)
0806 18:40:12.683290 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.683295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.696562 (+ 13267us) server_negotiation.cc:407] Connection header received
0806 18:40:12.696678 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.696682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.696754 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.696862 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.697839 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.698600 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.699687 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.700004 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.700106 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.701827 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.701831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.701834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.701862 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.703480 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.703963 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.703967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.703968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.704060 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.704320 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.704323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.704325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.704535 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.704661 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.704912 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.705115 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 18:40:12.705397 20150 connection.cc:535] server connection from 127.0.0.1:35640 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 18:40:12.705469 19943 connection.cc:535] server connection from 127.0.0.1:33628 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 18:40:12.705870 20369 connection.cc:535] server connection from 127.0.0.1:37422 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:40:12.716378 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.683618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.8.241.252:33933)
0806 18:40:12.683716 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.683720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.706157 (+ 22437us) server_negotiation.cc:407] Connection header received
0806 18:40:12.706385 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.706389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.706441 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.706493 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.707368 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.708115 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.709622 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.709984 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.710092 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.712094 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.712098 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.712101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.712130 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.713879 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.714334 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.714338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.714339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.714425 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.714733 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.714736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.714738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.715324 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.715476 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.716097 (+ 621us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.716213 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:40:12.719509 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.709597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.8.241.253:41923)
0806 18:40:12.709702 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.709707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.710584 (+ 877us) server_negotiation.cc:407] Connection header received
0806 18:40:12.710746 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.710749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.710792 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.710839 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.711796 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.712586 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.714039 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.714365 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.714433 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.716227 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.716231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.716234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.716260 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.717830 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.718275 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.718279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.718280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.718359 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.718638 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.718642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.718644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.718855 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.718967 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.719229 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.719354 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:40:12.719614 20561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:12.710318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59922 (local address 127.8.241.252:33933)
0806 18:40:12.710428 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:12.710432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:12.710455 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:40:12.710493 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:12.710495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:12.710546 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:12.710622 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:12.711628 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.712382 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.714059 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:12.714389 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:12.714462 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:12.716090 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:12.716093 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:12.716096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:12.716114 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:12.718009 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.718426 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.718430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.718432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.718516 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:12.718794 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:12.718798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:12.718799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:12.719015 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:12.719144 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:12.719343 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 18:40:12.719462 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 18:40:13.411830 20529 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.8.241.195:37419
I20260806 18:40:13.669549 20647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:13.668440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.195:57009)
0806 18:40:13.668718 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:13.669095 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260806 18:40:13.669605 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:13.669170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:34983)
0806 18:40:13.669390 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:13.669498 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":68,"threads_started":1}
W20260806 18:40:13.670182 20117 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:43515 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
W20260806 18:40:13.670297 20543 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:43515 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
I20260806 18:40:13.670701 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:13.670045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.194:44137)
0806 18:40:13.670503 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:13.670613 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":68,"threads_started":1}
W20260806 18:40:13.670943 20325 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:43515 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
I20260806 18:40:14.180944 20650 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc7173999834375bfaeaf8f77292c24)
I20260806 18:40:14.181080 20651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7bc7173999834375bfaeaf8f77292c24)
I20260806 18:40:14.181210 20650 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } }
I20260806 18:40:14.181283 20651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } }
I20260806 18:40:14.182214 20650 leader_election.cc:288] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515), 36971c9b70d741a58fecf5e5690873f0 (127.8.241.253:41923)
I20260806 18:40:14.182219 20651 leader_election.cc:288] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515), eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252:33933)
I20260806 18:40:14.182845 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.182362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57798)
0806 18:40:14.182650 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.182739 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":57,"threads_started":1}
I20260806 18:40:14.182951 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.182362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57796)
0806 18:40:14.182781 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.182856 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":73,"threads_started":1}
W20260806 18:40:14.183009 19510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
W20260806 18:40:14.183101 19815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
I20260806 18:40:14.183915 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.183683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57810)
0806 18:40:14.183812 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.183859 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 18:40:14.184020 19510 leader_election.cc:334] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515): Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
I20260806 18:40:14.184577 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.184356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57816)
0806 18:40:14.184458 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.184512 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:40:14.184773 19815 leader_election.cc:334] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515): Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
I20260806 18:40:14.186722 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.182870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:41923 (local address 127.0.0.1:41864)
0806 18:40:14.183089 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.183102 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:14.183209 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:14.183748 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:14.183752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:14.183766 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:14.183866 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:14.183873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:14.184816 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:14.184819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:14.185678 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:14.185688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:14.186315 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:14.186318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:14.186407 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:14.186430 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:40:14.186497 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:40:14.186571 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":3,"thread_start_us":259,"threads_started":1}
I20260806 18:40:14.186722 20659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.182663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41864 (local address 127.8.241.253:41923)
0806 18:40:14.183001 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:14.183005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:14.183532 (+ 527us) server_negotiation.cc:407] Connection header received
0806 18:40:14.183587 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:14.183590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:14.183640 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:14.183730 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:14.183991 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:14.184712 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:14.185797 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:14.186207 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:14.186301 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:14.186361 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:14.186521 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:40:14.186599 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":71,"threads_started":1}
I20260806 18:40:14.186753 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.182744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59932 (local address 127.8.241.252:33933)
0806 18:40:14.183477 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:14.183482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:14.183499 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:40:14.183626 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:14.183629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:14.183668 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:14.183733 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:14.184078 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:14.184741 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:14.185825 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:14.186230 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:14.186323 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:14.186381 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:14.186543 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 18:40:14.186619 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":224,"server-negotiator.queue_time_us":672,"thread_start_us":78,"threads_started":1}
I20260806 18:40:14.186753 20658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.182482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:33933 (local address 127.0.0.1:59932)
0806 18:40:14.183388 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.183401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:14.183499 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:14.183846 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:14.183849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:14.183866 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:14.183949 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:14.183956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:14.184862 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:14.184865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:14.185702 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:14.185713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:14.186340 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:14.186343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:14.186429 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:14.186452 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:40:14.186518 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:40:14.186593 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":58,"threads_started":1}
I20260806 18:40:14.187498 19535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb579dc4a65a4555b6c1011b3a874f74" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "36971c9b70d741a58fecf5e5690873f0" is_pre_election: true
I20260806 18:40:14.187572 19848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36971c9b70d741a58fecf5e5690873f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eb579dc4a65a4555b6c1011b3a874f74" is_pre_election: true
I20260806 18:40:14.187700 19535 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb579dc4a65a4555b6c1011b3a874f74 in term 1.
I20260806 18:40:14.187728 19848 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36971c9b70d741a58fecf5e5690873f0 in term 1.
I20260806 18:40:14.188061 19814 leader_election.cc:302] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36971c9b70d741a58fecf5e5690873f0, eb579dc4a65a4555b6c1011b3a874f74; no voters: 7bc7173999834375bfaeaf8f77292c24
I20260806 18:40:14.188124 19507 leader_election.cc:302] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36971c9b70d741a58fecf5e5690873f0, eb579dc4a65a4555b6c1011b3a874f74; no voters: 7bc7173999834375bfaeaf8f77292c24
I20260806 18:40:14.188292 20650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:40:14.188350 20650 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc7173999834375bfaeaf8f77292c24)
I20260806 18:40:14.188385 20650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:40:14.188441 20651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:40:14.188495 20651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7bc7173999834375bfaeaf8f77292c24)
I20260806 18:40:14.188521 20651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:40:14.189558 20651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } }
I20260806 18:40:14.189613 20650 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } }
I20260806 18:40:14.189881 20651 leader_election.cc:288] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515), eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252:33933)
I20260806 18:40:14.189944 20650 leader_election.cc:288] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [CANDIDATE]: Term 2 election: Requested vote from peers 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515), 36971c9b70d741a58fecf5e5690873f0 (127.8.241.253:41923)
I20260806 18:40:14.190267 20658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.190019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57822)
0806 18:40:14.190150 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.190205 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 18:40:14.190363 19535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb579dc4a65a4555b6c1011b3a874f74" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "36971c9b70d741a58fecf5e5690873f0"
I20260806 18:40:14.190363 19848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36971c9b70d741a58fecf5e5690873f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eb579dc4a65a4555b6c1011b3a874f74"
I20260806 18:40:14.190455 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.190140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57828)
0806 18:40:14.190270 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.190393 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:40:14.190537 19848 raft_consensus.cc:2379] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 36971c9b70d741a58fecf5e5690873f0 in current term 2: Already voted for candidate eb579dc4a65a4555b6c1011b3a874f74 in this term.
I20260806 18:40:14.190537 19535 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eb579dc4a65a4555b6c1011b3a874f74 in current term 2: Already voted for candidate 36971c9b70d741a58fecf5e5690873f0 in this term.
I20260806 18:40:14.191248 20658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.191055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57840)
0806 18:40:14.191141 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.191187 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 18:40:14.191360 20657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.191161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:57850)
0806 18:40:14.191256 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.191304 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:40:14.191360 19510 leader_election.cc:334] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515): Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
W20260806 18:40:14.191481 19815 leader_election.cc:334] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515): Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
I20260806 18:40:14.191484 19510 leader_election.cc:302] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 36971c9b70d741a58fecf5e5690873f0; no voters: 7bc7173999834375bfaeaf8f77292c24, eb579dc4a65a4555b6c1011b3a874f74
I20260806 18:40:14.191545 19815 leader_election.cc:302] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb579dc4a65a4555b6c1011b3a874f74; no voters: 36971c9b70d741a58fecf5e5690873f0, 7bc7173999834375bfaeaf8f77292c24
I20260806 18:40:14.191628 20651 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 18:40:14.191749 20650 raft_consensus.cc:2726] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 18:40:14.671382 20666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.670742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:36639)
0806 18:40:14.671193 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.671267 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":110,"threads_started":1}
I20260806 18:40:14.671831 20667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.671054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.195:49973)
0806 18:40:14.671674 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.671739 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":83,"threads_started":1}
I20260806 18:40:14.672098 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:14.671360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.194:38157)
0806 18:40:14.671941 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:14.672007 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":109,"threads_started":1}
I20260806 18:40:15.672611 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.672059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:47461)
0806 18:40:15.672392 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.672493 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":87,"threads_started":1}
I20260806 18:40:15.673058 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.672574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.195:36405)
0806 18:40:15.672858 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.672922 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":68,"threads_started":1}
I20260806 18:40:15.673465 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.672999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.194:45863)
0806 18:40:15.673301 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.673362 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":66,"threads_started":1}
I20260806 18:40:15.727289 20675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:15.727433 20675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } }
I20260806 18:40:15.727922 20675 leader_election.cc:288] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515), eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252:33933)
I20260806 18:40:15.728536 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.728135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:58050)
0806 18:40:15.728372 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.728446 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":62,"threads_started":1}
I20260806 18:40:15.728516 19848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36971c9b70d741a58fecf5e5690873f0" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eb579dc4a65a4555b6c1011b3a874f74" is_pre_election: true
I20260806 18:40:15.728663 19848 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36971c9b70d741a58fecf5e5690873f0 in term 2.
I20260806 18:40:15.729043 19507 leader_election.cc:302] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36971c9b70d741a58fecf5e5690873f0, eb579dc4a65a4555b6c1011b3a874f74; no voters:
I20260806 18:40:15.729241 20675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 18:40:15.729298 20675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:15.729336 20675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:40:15.729417 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.729152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:58052)
0806 18:40:15.729293 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.729349 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 18:40:15.729525 19510 leader_election.cc:334] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515): Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
I20260806 18:40:15.730424 20675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } }
I20260806 18:40:15.730741 20675 leader_election.cc:288] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 3 election: Requested vote from peers 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515), eb579dc4a65a4555b6c1011b3a874f74 (127.8.241.252:33933)
I20260806 18:40:15.731084 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.730894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:58058)
0806 18:40:15.730979 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.731026 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 18:40:15.731216 19848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36971c9b70d741a58fecf5e5690873f0" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "eb579dc4a65a4555b6c1011b3a874f74"
I20260806 18:40:15.731370 19848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:40:15.731731 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.731551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:58064)
0806 18:40:15.731641 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.731685 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 18:40:15.731861 19510 leader_election.cc:334] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515): Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
I20260806 18:40:15.732467 19848 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36971c9b70d741a58fecf5e5690873f0 in term 3.
I20260806 18:40:15.732735 19507 leader_election.cc:302] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 36971c9b70d741a58fecf5e5690873f0, eb579dc4a65a4555b6c1011b3a874f74; no voters: 7bc7173999834375bfaeaf8f77292c24
I20260806 18:40:15.732909 20675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:40:15.733104 20675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [term 3 LEADER]: Becoming Leader. State: Replica: 36971c9b70d741a58fecf5e5690873f0, State: Running, Role: LEADER
I20260806 18:40:15.733366 20675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } }
I20260806 18:40:15.734871 20680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36971c9b70d741a58fecf5e5690873f0. Latest consensus state: current_term: 3 leader_uuid: "36971c9b70d741a58fecf5e5690873f0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:15.734997 20680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:15.735285 20683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:15.737720 20683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:15.738135 20683 catalog_manager.cc:1259] Loaded cluster ID: e2853932c97c4ac2b942f359e8756eb2
I20260806 18:40:15.738180 20683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:15.738698 20683 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:15.739084 20683 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Loaded TSK: 0
I20260806 18:40:15.739750 20683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:40:15.765681 19522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:40:15.768982 19522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:15.772082 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.771846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:58072)
0806 18:40:15.771958 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.772012 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 18:40:15.772209 19848 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [term 3 FOLLOWER]: Refusing update from remote peer 36971c9b70d741a58fecf5e5690873f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 18:40:15.772770 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.772549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.0.0.1:58076)
0806 18:40:15.772662 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.772712 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 18:40:15.772737 20680 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 18:40:15.772892 19510 consensus_peers.cc:487] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 -> Peer 7bc7173999834375bfaeaf8f77292c24 (127.8.241.254:43515): Couldn't send request to peer 7bc7173999834375bfaeaf8f77292c24. Status: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 18:40:15.774315 20686 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36971c9b70d741a58fecf5e5690873f0. Latest consensus state: current_term: 3 leader_uuid: "36971c9b70d741a58fecf5e5690873f0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:15.774456 20686 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:40:15.775615 20680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "36971c9b70d741a58fecf5e5690873f0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:15.775745 20680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:15.776280 20687 mvcc.cc:205] Tried to move back new op lower bound from 7315626458198044672 to 7315626458046275584. Current Snapshot: MvccSnapshot[applied={T|T < 7315626458198044672 or (T in {7315626458198044672})}]
I20260806 18:40:15.776559 20686 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "36971c9b70d741a58fecf5e5690873f0" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "7bc7173999834375bfaeaf8f77292c24" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43515 } } peers { permanent_uuid: "36971c9b70d741a58fecf5e5690873f0" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 41923 } attrs { promote: false } } peers { permanent_uuid: "eb579dc4a65a4555b6c1011b3a874f74" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 33933 } attrs { promote: false } } }
I20260806 18:40:15.778353 20686 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb579dc4a65a4555b6c1011b3a874f74 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:40:15.786690 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.781922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37419 (local address 127.0.0.1:37428)
0806 18:40:15.782311 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.782327 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:15.782422 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:15.783207 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:15.783210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:15.783235 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:15.783311 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.783318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.784294 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.784297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.785420 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.785430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.786283 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.786286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.786371 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.786400 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:40:15.786475 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:40:15.786547 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":39}
I20260806 18:40:15.786728 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.782049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37428 (local address 127.8.241.195:37419)
0806 18:40:15.782958 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:15.782963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:15.782976 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:15.783026 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:15.783028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:15.783072 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:15.783195 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:15.783418 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.784185 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.785537 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.786177 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.786274 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.786340 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:15.786500 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:40:15.786580 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":825,"thread_start_us":87,"threads_started":1}
I20260806 18:40:15.786998 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.782531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44305 (local address 127.0.0.1:33634)
0806 18:40:15.783170 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.783183 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:15.783265 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:15.783545 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:15.783548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:15.783565 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:15.783655 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.783661 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.784523 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.784526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.785443 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.785454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.786401 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.786405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.786491 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.786515 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:40:15.786579 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:40:15.786640 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"mutex_wait_us":17,"thread_start_us":90,"threads_started":1}
I20260806 18:40:15.787245 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.782449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.8.241.194:39639)
0806 18:40:15.782715 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:15.782721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:15.782755 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:40:15.782812 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:15.782815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:15.782866 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:15.782962 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:15.783283 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.783975 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.786039 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.786588 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.786679 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.786741 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:15.787029 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 18:40:15.787110 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":88,"threads_started":1}
I20260806 18:40:15.787298 20441 tablet_service.cc:1467] Processing CreateTablet for tablet f911917f887f43c088a0251d84f17b5e (DEFAULT_TABLE table=test-table [id=f9f3b7c6045643ecbc0bbc5e0f8e948d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:15.787433 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.782901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33634 (local address 127.8.241.193:44305)
0806 18:40:15.783145 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:15.783149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:15.783338 (+ 189us) server_negotiation.cc:407] Connection header received
0806 18:40:15.783393 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:15.783396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:15.783445 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:15.783528 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:15.783756 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.784426 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.785578 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.786169 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.786274 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.786340 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:15.786751 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 18:40:15.787310 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260806 18:40:15.787611 20441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f911917f887f43c088a0251d84f17b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:15.787642 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.782198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:39639 (local address 127.0.0.1:35664)
0806 18:40:15.782604 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.782616 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:15.782692 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:15.782990 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:15.782993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:15.783024 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:15.783138 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.783145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.784085 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.784087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.785922 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.785932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.786802 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.786805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.786885 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.786910 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:40:15.787490 (+ 580us) client_negotiation.cc:240] Negotiation successful
0806 18:40:15.787553 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":23,"thread_start_us":97,"threads_started":1}
I20260806 18:40:15.787957 20006 tablet_service.cc:1467] Processing CreateTablet for tablet f911917f887f43c088a0251d84f17b5e (DEFAULT_TABLE table=test-table [id=f9f3b7c6045643ecbc0bbc5e0f8e948d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:15.788206 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f911917f887f43c088a0251d84f17b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:15.788180 20226 tablet_service.cc:1467] Processing CreateTablet for tablet f911917f887f43c088a0251d84f17b5e (DEFAULT_TABLE table=test-table [id=f9f3b7c6045643ecbc0bbc5e0f8e948d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:15.788417 20226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f911917f887f43c088a0251d84f17b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:15.790217 20698 tablet_bootstrap.cc:492] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: Bootstrap starting.
I20260806 18:40:15.790827 20699 tablet_bootstrap.cc:492] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f: Bootstrap starting.
I20260806 18:40:15.791103 20698 tablet_bootstrap.cc:654] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:15.791136 20701 tablet_bootstrap.cc:492] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: Bootstrap starting.
I20260806 18:40:15.791705 20699 tablet_bootstrap.cc:654] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:15.792019 20701 tablet_bootstrap.cc:654] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:15.792323 20698 tablet_bootstrap.cc:492] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: No bootstrap required, opened a new log
I20260806 18:40:15.792466 20698 ts_tablet_manager.cc:1397] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:15.792850 20699 tablet_bootstrap.cc:492] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f: No bootstrap required, opened a new log
I20260806 18:40:15.792948 20699 ts_tablet_manager.cc:1397] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:15.792966 20698 raft_consensus.cc:348] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } }
I20260806 18:40:15.793123 20698 raft_consensus.cc:374] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:15.793165 20698 raft_consensus.cc:729] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bd32d76dd24d43b2bc5aacc4e72f3b, State: Initialized, Role: FOLLOWER
I20260806 18:40:15.793305 20698 consensus_queue.cc:260] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } }
I20260806 18:40:15.793366 20699 raft_consensus.cc:348] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } }
I20260806 18:40:15.793473 20699 raft_consensus.cc:374] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:15.793514 20699 raft_consensus.cc:729] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6ff04d70057445c801791a432a44a1f, State: Initialized, Role: FOLLOWER
I20260806 18:40:15.793663 20698 ts_tablet_manager.cc:1428] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:15.793648 20699 consensus_queue.cc:260] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } }
I20260806 18:40:15.793998 20699 ts_tablet_manager.cc:1428] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:15.794307 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.794009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.195:38851)
0806 18:40:15.794163 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.794223 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 18:40:15.794970 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.794735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.195:39373)
0806 18:40:15.794865 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.794911 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 18:40:15.795928 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.795659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:60461)
0806 18:40:15.795794 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.795856 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:40:15.796687 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.796444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:36111)
0806 18:40:15.796581 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.796630 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 18:40:15.796759 20118 heartbeater.cc:503] Master 127.8.241.253:41923 was elected leader, sending a full tablet report...
I20260806 18:40:15.797120 20545 heartbeater.cc:503] Master 127.8.241.253:41923 was elected leader, sending a full tablet report...
I20260806 18:40:15.797932 20701 tablet_bootstrap.cc:492] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: No bootstrap required, opened a new log
I20260806 18:40:15.798038 20701 ts_tablet_manager.cc:1397] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260806 18:40:15.798516 20701 raft_consensus.cc:348] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } }
I20260806 18:40:15.798617 20701 raft_consensus.cc:374] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:15.798638 20701 raft_consensus.cc:729] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ac15e9acfb46c08395ce0aa8819ccc, State: Initialized, Role: FOLLOWER
I20260806 18:40:15.798766 20701 consensus_queue.cc:260] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } }
I20260806 18:40:15.799115 20701 ts_tablet_manager.cc:1428] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:15.799705 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.799497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.194:59111)
0806 18:40:15.799592 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.799645 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 18:40:15.800477 20327 heartbeater.cc:503] Master 127.8.241.253:41923 was elected leader, sending a full tablet report...
I20260806 18:40:15.800835 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.800611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.194:43923)
0806 18:40:15.800737 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.800776 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 18:40:15.823333 20707 raft_consensus.cc:482] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:15.823499 20707 raft_consensus.cc:504] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } }
I20260806 18:40:15.824589 20707 leader_election.cc:288] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79bd32d76dd24d43b2bc5aacc4e72f3b (127.8.241.195:37419), 87ac15e9acfb46c08395ce0aa8819ccc (127.8.241.194:39639)
I20260806 18:40:15.828438 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.824846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37419 (local address 127.8.241.193:48499)
0806 18:40:15.825097 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.825112 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:15.825237 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:15.825530 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:15.825533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:15.825551 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:15.825691 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.825699 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.826485 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.826489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.827331 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.827341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.827993 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.827996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.828128 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.828157 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:40:15.828227 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:40:15.828296 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":50}
I20260806 18:40:15.828469 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.824975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:60107 (local address 127.8.241.194:39639)
0806 18:40:15.825130 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:15.825135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:15.825431 (+ 296us) server_negotiation.cc:407] Connection header received
0806 18:40:15.825506 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:15.825509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:15.825561 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:15.825645 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:15.825912 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.826408 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.827471 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.827854 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.827976 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.828033 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:15.828286 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 18:40:15.828365 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 18:40:15.828460 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.824922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:48499 (local address 127.8.241.195:37419)
0806 18:40:15.825078 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:15.825083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:15.825190 (+ 107us) server_negotiation.cc:407] Connection header received
0806 18:40:15.825349 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:15.825353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:15.825408 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:15.825482 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:15.825828 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.826357 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.827457 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.827846 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.827957 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.828015 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:15.828262 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 18:40:15.828340 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:40:15.828750 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.824862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:39639 (local address 127.8.241.193:60107)
0806 18:40:15.825309 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.825322 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:15.825425 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:15.825671 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:15.825673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:15.825689 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:15.825801 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.825808 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.826519 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.826522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.827342 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:15.827352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.827988 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.827992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:15.828141 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.828165 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:40:15.828452 (+ 287us) client_negotiation.cc:240] Negotiation successful
0806 18:40:15.828528 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":92,"thread_start_us":150,"threads_started":1}
I20260806 18:40:15.829094 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f911917f887f43c088a0251d84f17b5e" candidate_uuid: "c6ff04d70057445c801791a432a44a1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" is_pre_election: true
I20260806 18:40:15.829267 20472 raft_consensus.cc:2454] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6ff04d70057445c801791a432a44a1f in term 0.
I20260806 18:40:15.829445 20258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f911917f887f43c088a0251d84f17b5e" candidate_uuid: "c6ff04d70057445c801791a432a44a1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" is_pre_election: true
I20260806 18:40:15.829705 20258 raft_consensus.cc:2454] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6ff04d70057445c801791a432a44a1f in term 0.
I20260806 18:40:15.829682 19942 leader_election.cc:302] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79bd32d76dd24d43b2bc5aacc4e72f3b, c6ff04d70057445c801791a432a44a1f; no voters:
I20260806 18:40:15.830071 20707 raft_consensus.cc:2781] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:15.830144 20707 raft_consensus.cc:482] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:15.830180 20707 raft_consensus.cc:3037] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:15.831199 20707 raft_consensus.cc:504] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } }
I20260806 18:40:15.831533 20707 leader_election.cc:288] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [CANDIDATE]: Term 1 election: Requested vote from peers 79bd32d76dd24d43b2bc5aacc4e72f3b (127.8.241.195:37419), 87ac15e9acfb46c08395ce0aa8819ccc (127.8.241.194:39639)
I20260806 18:40:15.831888 20258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f911917f887f43c088a0251d84f17b5e" candidate_uuid: "c6ff04d70057445c801791a432a44a1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87ac15e9acfb46c08395ce0aa8819ccc"
I20260806 18:40:15.831902 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f911917f887f43c088a0251d84f17b5e" candidate_uuid: "c6ff04d70057445c801791a432a44a1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b"
I20260806 18:40:15.832003 20258 raft_consensus.cc:3037] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:15.832080 20472 raft_consensus.cc:3037] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:15.833048 20258 raft_consensus.cc:2454] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6ff04d70057445c801791a432a44a1f in term 1.
I20260806 18:40:15.833130 20472 raft_consensus.cc:2454] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6ff04d70057445c801791a432a44a1f in term 1.
I20260806 18:40:15.833391 19941 leader_election.cc:302] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87ac15e9acfb46c08395ce0aa8819ccc, c6ff04d70057445c801791a432a44a1f; no voters:
I20260806 18:40:15.833570 20707 raft_consensus.cc:2781] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:15.833794 20707 raft_consensus.cc:686] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 1 LEADER]: Becoming Leader. State: Replica: c6ff04d70057445c801791a432a44a1f, State: Running, Role: LEADER
I20260806 18:40:15.834050 20707 consensus_queue.cc:237] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } }
I20260806 18:40:15.834481 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.834256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:48219)
0806 18:40:15.834357 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.834410 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 18:40:15.836549 19528 catalog_manager.cc:5495] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f reported cstate change: term changed from 0 to 1, leader changed from <none> to c6ff04d70057445c801791a432a44a1f (127.8.241.193). New cstate: current_term: 1 leader_uuid: "c6ff04d70057445c801791a432a44a1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6ff04d70057445c801791a432a44a1f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44305 } health_report { overall_health: HEALTHY } } }
I20260806 18:40:15.849958 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.846526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.8.241.193:44305)
0806 18:40:15.846638 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:15.846642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:15.846655 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:15.846797 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:15.846799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:15.846853 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:15.846942 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:15.847185 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.847692 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.848522 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:15.848734 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:15.848820 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:15.849114 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:15.849574 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:15.849677 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:15.849767 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 18:40:15.849824 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:40:15.852763 20472 raft_consensus.cc:1261] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 1 FOLLOWER]: Refusing update from remote peer c6ff04d70057445c801791a432a44a1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:15.853081 20258 raft_consensus.cc:1261] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [term 1 FOLLOWER]: Refusing update from remote peer c6ff04d70057445c801791a432a44a1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:15.853335 20707 consensus_queue.cc:1035] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "79bd32d76dd24d43b2bc5aacc4e72f3b" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:15.853648 20711 consensus_queue.cc:1035] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "87ac15e9acfb46c08395ce0aa8819ccc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:15.855754 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.855437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.195:44369)
0806 18:40:15.855606 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.855670 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260806 18:40:15.855839 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.855437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.194:37185)
0806 18:40:15.855700 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.855763 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260806 18:40:15.859113 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.858828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:40869)
0806 18:40:15.858965 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.859032 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 18:40:15.860661 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.860394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.195:35547)
0806 18:40:15.860526 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.860581 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 18:40:15.860895 20710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.860662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.193:43653)
0806 18:40:15.860768 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.860825 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:40:15.862521 20673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.862162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43515 (local address 127.8.241.194:45463)
0806 18:40:15.862387 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.862448 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260806 18:40:15.870795 19522 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:15.870960 19522 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:15.875415 19522 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36971c9b70d741a58fecf5e5690873f0: Sending DeleteTablet for 3 replicas of tablet f911917f887f43c088a0251d84f17b5e
I20260806 18:40:15.876174 20441 tablet_service.cc:1514] Processing DeleteTablet for tablet f911917f887f43c088a0251d84f17b5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:15 UTC) from {username='oryx', principal='oryx/127.8.241.253@KRBTEST.COM'} at 127.0.0.1:37428
I20260806 18:40:15.876324 20226 tablet_service.cc:1514] Processing DeleteTablet for tablet f911917f887f43c088a0251d84f17b5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:15 UTC) from {username='oryx', principal='oryx/127.8.241.253@KRBTEST.COM'} at 127.0.0.1:35664
I20260806 18:40:15.876735 20735 tablet_replica.cc:330] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: stopping tablet replica
I20260806 18:40:15.876735 20006 tablet_service.cc:1514] Processing DeleteTablet for tablet f911917f887f43c088a0251d84f17b5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:15 UTC) from {username='oryx', principal='oryx/127.8.241.253@KRBTEST.COM'} at 127.0.0.1:33634
I20260806 18:40:15.876960 20735 raft_consensus.cc:2229] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:15.877126 20738 tablet_replica.cc:330] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f: stopping tablet replica
I20260806 18:40:15.877187 20735 raft_consensus.cc:2258] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:15.877307 20738 raft_consensus.cc:2229] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:40:15.877635 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 19770
I20260806 18:40:15.877691 20738 raft_consensus.cc:2258] T f911917f887f43c088a0251d84f17b5e P c6ff04d70057445c801791a432a44a1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:15.880420 20735 ts_tablet_manager.cc:1905] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:15.883792 20735 ts_tablet_manager.cc:1918] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:15.883882 20735 log.cc:1196] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/f911917f887f43c088a0251d84f17b5e
I20260806 18:40:15.884243 20735 ts_tablet_manager.cc:1939] T f911917f887f43c088a0251d84f17b5e P 87ac15e9acfb46c08395ce0aa8819ccc: Deleting consensus metadata
I20260806 18:40:15.885149 19509 catalog_manager.cc:4841] TS 87ac15e9acfb46c08395ce0aa8819ccc (127.8.241.194:39639): tablet f911917f887f43c088a0251d84f17b5e (table test-table [id=f9f3b7c6045643ecbc0bbc5e0f8e948d]) successfully deleted
I20260806 18:40:15.893311 20734 tablet_replica.cc:330] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: stopping tablet replica
I20260806 18:40:15.893481 20734 raft_consensus.cc:2229] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:15.893666 20734 raft_consensus.cc:2258] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:15.894485 20734 ts_tablet_manager.cc:1905] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:15.896982 20734 ts_tablet_manager.cc:1918] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:15.897190 20734 log.cc:1196] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/f911917f887f43c088a0251d84f17b5e
I20260806 18:40:15.897212 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 20128
I20260806 18:40:15.897387 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:15.897103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44305 (local address 127.0.0.1:33664)
0806 18:40:15.897236 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:15.897294 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:40:15.897473 20734 ts_tablet_manager.cc:1939] T f911917f887f43c088a0251d84f17b5e P 79bd32d76dd24d43b2bc5aacc4e72f3b: Deleting consensus metadata
W20260806 18:40:15.901554 19509 catalog_manager.cc:4584] TS c6ff04d70057445c801791a432a44a1f (127.8.241.193:44305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f911917f887f43c088a0251d84f17b5e: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44305: connect: Connection refused (error 111)
I20260806 18:40:15.901584 19510 catalog_manager.cc:4841] TS 79bd32d76dd24d43b2bc5aacc4e72f3b (127.8.241.195:37419): tablet f911917f887f43c088a0251d84f17b5e (table test-table [id=f9f3b7c6045643ecbc0bbc5e0f8e948d]) successfully deleted
I20260806 18:40:15.917222 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 20336
I20260806 18:40:15.939338 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 19287
I20260806 18:40:15.948359 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 19492
2026-08-06T18:40:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7035 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:40:16 dist-test-slave-k5rr krb5kdc[20741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:40:16 dist-test-slave-k5rr krb5kdc[20741](info): set up 2 sockets
Aug 06 18:40:16 dist-test-slave-k5rr krb5kdc[20741](info): commencing operation
krb5kdc: starting...
W20260806 18:40:17.086968 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:40:17 dist-test-slave-k5rr krb5kdc[20741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:40:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.254.keytab.
Entry for principal oryx/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.254.keytab.
I20260806 18:40:17.205255 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36675
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.254.keytab
--principal=oryx/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:36675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:17.316110 20760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:17.316366 20760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:17.316406 20760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:17.320345 20760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:17.320400 20760 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:40:17.320415 20760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:17.320434 20760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:17.320464 20760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:17.320492 20760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:17.325690 20760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:36675
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.254.keytab
--principal=oryx/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:17.326768 20760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:17.327801 20760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:17.336318 20769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:17.336380 20773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:17.336380 20771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:17.336555 20760 server_base.cc:1034] running on GCE node
I20260806 18:40:17.337213 20760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:17.338333 20760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:17.339498 20760 hybrid_clock.cc:648] HybridClock initialized: now 1786041617339469 us; error 44 us; skew 500 ppm
Aug 06 18:40:17 dist-test-slave-k5rr krb5kdc[20741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041617, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:17.342181 20760 init.cc:377] Logged in from keytab as oryx/127.8.241.254@KRBTEST.COM (short username oryx)
I20260806 18:40:17.343061 20760 webserver.cc:460] Webserver started at http://127.8.241.254:39951/ using document root <none> and password file <none>
I20260806 18:40:17.343622 20760 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:17.343689 20760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:17.343883 20760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:17.346010 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "7c2dd4e5bc874757b5d5667f117b8e95"
format_stamp: "Formatted at 2026-08-06 18:40:17 on dist-test-slave-k5rr"
server_key: "8eee52c715805a7ffd64b58ca5f002ce"
server_key_iv: "05f53d9f1e85f44f5a355391b3d050b9"
server_key_version: "encryptionkey@0"
I20260806 18:40:17.346562 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "7c2dd4e5bc874757b5d5667f117b8e95"
format_stamp: "Formatted at 2026-08-06 18:40:17 on dist-test-slave-k5rr"
server_key: "8eee52c715805a7ffd64b58ca5f002ce"
server_key_iv: "05f53d9f1e85f44f5a355391b3d050b9"
server_key_version: "encryptionkey@0"
I20260806 18:40:17.350440 20760 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:40:17.352761 20783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:17.353986 20760 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:17.354104 20760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "7c2dd4e5bc874757b5d5667f117b8e95"
format_stamp: "Formatted at 2026-08-06 18:40:17 on dist-test-slave-k5rr"
server_key: "8eee52c715805a7ffd64b58ca5f002ce"
server_key_iv: "05f53d9f1e85f44f5a355391b3d050b9"
server_key_version: "encryptionkey@0"
I20260806 18:40:17.354197 20760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:17.387362 20760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:17.390349 20760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:17.390547 20760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:17.398615 20760 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:36675
I20260806 18:40:17.398623 20876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:36675 every 8 connection(s)
I20260806 18:40:17.399612 20760 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:17.401270 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 20760
I20260806 18:40:17.401396 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:17.401676 9159 external_mini_cluster.cc:1428] Setting key a4c478ed3faa7055d74e9fa68fda28e4
I20260806 18:40:17.402861 20878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:17.409350 20878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95: Bootstrap starting.
Aug 06 18:40:17 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:17.411882 20878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:17.413079 20878 log.cc:824] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:17.415006 20878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95: No bootstrap required, opened a new log
I20260806 18:40:17.415560 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:17.403069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52472 (local address 127.8.241.254:36675)
0806 18:40:17.403503 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:17.403520 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:17.403555 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:40:17.404192 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:17.404206 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:17.404538 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:17.404812 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:17.405748 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.406729 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.407809 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.408254 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.408361 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:17.411112 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:17.411117 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:17.411130 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:17.411157 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:17.412672 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.413155 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.413160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.413164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.413296 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.413610 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.413613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.413614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.414028 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:17.414207 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:17.414512 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 18:40:17.414787 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":117,"threads_started":1}
I20260806 18:40:17.418009 20878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2dd4e5bc874757b5d5667f117b8e95" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36675 } }
I20260806 18:40:17.418236 20878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:17.418287 20878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2dd4e5bc874757b5d5667f117b8e95, State: Initialized, Role: FOLLOWER
I20260806 18:40:17.418672 20878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2dd4e5bc874757b5d5667f117b8e95" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36675 } }
I20260806 18:40:17.418797 20878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:17.418855 20878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:17.418956 20878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:17.419763 20878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2dd4e5bc874757b5d5667f117b8e95" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36675 } }
I20260806 18:40:17.420087 20878 leader_election.cc:302] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c2dd4e5bc874757b5d5667f117b8e95; no voters:
I20260806 18:40:17.420373 20878 leader_election.cc:288] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:17.420473 20885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:17.420708 20885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [term 1 LEADER]: Becoming Leader. State: Replica: 7c2dd4e5bc874757b5d5667f117b8e95, State: Running, Role: LEADER
I20260806 18:40:17.421072 20885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2dd4e5bc874757b5d5667f117b8e95" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36675 } }
I20260806 18:40:17.421566 20878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:17.422698 20888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2dd4e5bc874757b5d5667f117b8e95. Latest consensus state: current_term: 1 leader_uuid: "7c2dd4e5bc874757b5d5667f117b8e95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2dd4e5bc874757b5d5667f117b8e95" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36675 } } }
I20260806 18:40:17.422825 20888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:17.423171 20899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:17.423197 20887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c2dd4e5bc874757b5d5667f117b8e95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c2dd4e5bc874757b5d5667f117b8e95" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36675 } } }
I20260806 18:40:17.423872 20887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:17.426537 20899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:17.432518 20899 catalog_manager.cc:1347] Generated new cluster ID: ddda72db173a4754b68480bb005dd257
I20260806 18:40:17.432583 20899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:17.449113 20899 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:17.449992 20899 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:17.458014 20899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95: Generated new TSK 0
I20260806 18:40:17.458689 20899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab.
Entry for principal oryx/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab.
I20260806 18:40:17.507364 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:36675
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab
--principal=oryx/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:17.641489 20918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:17.641738 20918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:17.641781 20918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:17.645756 20918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:17.645812 20918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:17.645849 20918 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:40:17.645882 20918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:17.650939 20918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab
--principal=oryx/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.241.254:36675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:17.651907 20918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:17.652868 20918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:17.662550 20926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:17.662614 20918 server_base.cc:1034] running on GCE node
W20260806 18:40:17.662550 20928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:17.662555 20924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:17.663188 20918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:17.663776 20918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:17.664930 20918 hybrid_clock.cc:648] HybridClock initialized: now 1786041617664911 us; error 29 us; skew 500 ppm
Aug 06 18:40:17 dist-test-slave-k5rr krb5kdc[20741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041617, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:17.667608 20918 init.cc:377] Logged in from keytab as oryx/127.8.241.193@KRBTEST.COM (short username oryx)
I20260806 18:40:17.668426 20918 webserver.cc:460] Webserver started at http://127.8.241.193:39711/ using document root <none> and password file <none>
I20260806 18:40:17.668960 20918 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:17.669057 20918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:17.669276 20918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:17.671372 20918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "b50a858476fb4c17a9d15a34b0d7028a"
format_stamp: "Formatted at 2026-08-06 18:40:17 on dist-test-slave-k5rr"
server_key: "db602137e8fc0a814ef633d0aad3a071"
server_key_iv: "a5e1b5c2a4a7e41de0e49d1deb5cca59"
server_key_version: "encryptionkey@0"
I20260806 18:40:17.671962 20918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "b50a858476fb4c17a9d15a34b0d7028a"
format_stamp: "Formatted at 2026-08-06 18:40:17 on dist-test-slave-k5rr"
server_key: "db602137e8fc0a814ef633d0aad3a071"
server_key_iv: "a5e1b5c2a4a7e41de0e49d1deb5cca59"
server_key_version: "encryptionkey@0"
I20260806 18:40:17.675714 20918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 18:40:17.678251 20940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:17.679570 20918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:40:17.679679 20918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "b50a858476fb4c17a9d15a34b0d7028a"
format_stamp: "Formatted at 2026-08-06 18:40:17 on dist-test-slave-k5rr"
server_key: "db602137e8fc0a814ef633d0aad3a071"
server_key_iv: "a5e1b5c2a4a7e41de0e49d1deb5cca59"
server_key_version: "encryptionkey@0"
I20260806 18:40:17.679788 20918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:17.705690 20918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:17.708592 20918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:17.708817 20918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:17.710376 20918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:17.710426 20918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:17.710470 20918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:17.710498 20918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:40:17 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041617, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:17.720783 20918 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:34265
I20260806 18:40:17.720877 21115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:34265 every 8 connection(s)
I20260806 18:40:17.721899 20918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:17.725263 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 20918
I20260806 18:40:17.725306 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:17.711274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:59569 (local address 127.8.241.254:36675)
0806 18:40:17.711392 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:17.711396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:17.712603 (+ 1207us) server_negotiation.cc:407] Connection header received
0806 18:40:17.713813 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:17.713817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:17.713879 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:17.713941 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:17.715659 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.716199 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.717895 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.718247 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.718391 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:17.721539 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:17.721543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:17.721545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:17.721573 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:17.723193 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.723737 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.723741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.723743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.723829 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.724147 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.724151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.724152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.724337 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:17.724422 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:17.724943 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 18:40:17.725136 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:40:17.725396 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:17.725719 9159 external_mini_cluster.cc:1428] Setting key f14a0b1dc2d620ab64dc19fa80f98a5b
I20260806 18:40:17.726091 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:17.711875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36675 (local address 127.8.241.193:59569)
0806 18:40:17.712428 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:17.712472 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:17.713515 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:17.714390 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:17.714399 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:17.714816 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:17.715514 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:17.715534 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.716337 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.716341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:17.717753 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:17.717765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.718461 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.718465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:17.718621 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:17.719361 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:17.719385 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:17.721339 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:17.723299 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:17.723305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:17.723321 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:17.723628 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:17.723910 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:17.723914 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:17.723916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:17.724051 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:17.724436 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:17.724442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:17.724699 (+ 257us) client_negotiation.cc:769] Sending connection context
0806 18:40:17.724863 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 18:40:17.725247 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":142,"threads_started":1}
I20260806 18:40:17.727391 21116 heartbeater.cc:347] Connected to a master server at 127.8.241.254:36675
I20260806 18:40:17.727618 21116 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:17.728099 21116 heartbeater.cc:511] Master 127.8.241.254:36675 requested a full tablet report, sending...
I20260806 18:40:17.730015 20809 ts_manager.cc:194] Registered new tserver with Master: b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265)
I20260806 18:40:17.731256 20809 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:59569
WARNING: no policy specified for oryx/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:40:17.737922 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:17.729980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:39361 (local address 127.8.241.254:36675)
0806 18:40:17.730091 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:17.730095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:17.731241 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 18:40:17.731424 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:17.731427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:17.731500 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:17.731555 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:17.732206 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.732876 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.733746 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.733965 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.734048 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:17.735109 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:17.735113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:17.735115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:17.735140 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:17.736398 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.736790 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.736794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.736796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.736868 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.737168 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.737171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.737173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.737344 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:17.737428 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:17.737672 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 18:40:17.737774 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:40:17.737946 21127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:17.729906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36675 (local address 127.8.241.193:39361)
0806 18:40:17.731159 (+ 1253us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:17.731174 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:17.731316 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:17.731722 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:17.731725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:17.731947 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:17.732086 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:17.732094 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.733042 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.733045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:17.733638 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:17.733648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.734065 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.734068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:17.734159 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:17.734641 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:17.734655 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:17.734983 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:17.736501 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:17.736505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:17.736507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:17.736697 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:17.736950 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:17.736953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:17.736955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:17.737079 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:17.737432 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:17.737436 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:17.737523 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 18:40:17.737610 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:40:17.737746 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1164,"mutex_wait_us":3,"thread_start_us":1060,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab.
Entry for principal oryx/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab.
I20260806 18:40:17.753974 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:36675
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab
--principal=oryx/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:17.867400 21134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:17.867666 21134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:17.867726 21134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:17.871805 21134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:17.871870 21134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:17.871924 21134 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:40:17.871981 21134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:17.877331 21134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab
--principal=oryx/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.241.254:36675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:17.878608 21134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:17.879729 21134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:40:17.889670 21134 server_base.cc:1034] running on GCE node
W20260806 18:40:17.889600 21140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:17.889602 21141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:17.889596 21144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:17.890311 21134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:17.890954 21134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:17.892108 21134 hybrid_clock.cc:648] HybridClock initialized: now 1786041617892084 us; error 36 us; skew 500 ppm
Aug 06 18:40:17 dist-test-slave-k5rr krb5kdc[20741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041617, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:17.894755 21134 init.cc:377] Logged in from keytab as oryx/127.8.241.194@KRBTEST.COM (short username oryx)
I20260806 18:40:17.895643 21134 webserver.cc:460] Webserver started at http://127.8.241.194:45989/ using document root <none> and password file <none>
I20260806 18:40:17.896206 21134 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:17.896270 21134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:17.896476 21134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:17.898623 21134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "f2b18e54b70a4fdda56847238660aabc"
format_stamp: "Formatted at 2026-08-06 18:40:17 on dist-test-slave-k5rr"
server_key: "dc2a5476da3615f6aa46d62e11ce211b"
server_key_iv: "4b15086d3321c6b6a55fe878cee830b9"
server_key_version: "encryptionkey@0"
I20260806 18:40:17.899179 21134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "f2b18e54b70a4fdda56847238660aabc"
format_stamp: "Formatted at 2026-08-06 18:40:17 on dist-test-slave-k5rr"
server_key: "dc2a5476da3615f6aa46d62e11ce211b"
server_key_iv: "4b15086d3321c6b6a55fe878cee830b9"
server_key_version: "encryptionkey@0"
I20260806 18:40:17.902895 21134 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 18:40:17.905294 21152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:17.906484 21134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:40:17.906600 21134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "f2b18e54b70a4fdda56847238660aabc"
format_stamp: "Formatted at 2026-08-06 18:40:17 on dist-test-slave-k5rr"
server_key: "dc2a5476da3615f6aa46d62e11ce211b"
server_key_iv: "4b15086d3321c6b6a55fe878cee830b9"
server_key_version: "encryptionkey@0"
I20260806 18:40:17.906697 21134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:17.928399 21134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:17.931638 21134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:17.931831 21134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:17.933435 21134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:17.933491 21134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:17.933548 21134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:17.933573 21134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:40:17 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041617, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:17.944233 21134 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:42267
I20260806 18:40:17.944342 21318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:42267 every 8 connection(s)
I20260806 18:40:17.945500 21134 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:17.946969 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:17.934250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:35885 (local address 127.8.241.254:36675)
0806 18:40:17.934362 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:17.934365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:17.935481 (+ 1116us) server_negotiation.cc:407] Connection header received
0806 18:40:17.936513 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:17.936517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:17.936576 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:17.936634 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:17.938208 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.938833 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.940067 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.940313 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.940409 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:17.943326 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:17.943330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:17.943333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:17.943360 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:17.944853 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.945376 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.945380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.945382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.945492 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.945845 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.945849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.945851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.946090 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:17.946188 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:17.946738 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 18:40:17.946845 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:40:17.947723 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:17.934837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36675 (local address 127.8.241.194:35885)
0806 18:40:17.935339 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:17.935384 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:17.936267 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:17.937052 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:17.937062 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:17.937519 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:17.938058 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:17.938073 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.938963 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.938966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:17.939952 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:17.939964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.940487 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.940491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:17.940684 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:17.941314 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:17.941335 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:17.943158 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:17.944964 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:17.944969 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:17.944981 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:17.945274 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:17.945589 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:17.945591 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:17.945593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:17.945737 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:17.946246 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:17.946252 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:17.946502 (+ 250us) client_negotiation.cc:769] Sending connection context
0806 18:40:17.946658 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 18:40:17.946906 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":140,"threads_started":1}
I20260806 18:40:17.948868 21319 heartbeater.cc:347] Connected to a master server at 127.8.241.254:36675
I20260806 18:40:17.949035 21319 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:17.949543 21319 heartbeater.cc:511] Master 127.8.241.254:36675 requested a full tablet report, sending...
I20260806 18:40:17.950623 20807 ts_manager.cc:194] Registered new tserver with Master: f2b18e54b70a4fdda56847238660aabc (127.8.241.194:42267)
I20260806 18:40:17.950902 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 21134
I20260806 18:40:17.951012 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:40:17.951334 9159 external_mini_cluster.cc:1428] Setting key f6007e5cf01c3fdc806cfc043be40b31
I20260806 18:40:17.951385 20807 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:35885
I20260806 18:40:17.958731 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:17.951078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:59055 (local address 127.8.241.254:36675)
0806 18:40:17.951201 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:17.951205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:17.951608 (+ 403us) server_negotiation.cc:407] Connection header received
0806 18:40:17.951995 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:17.951998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:17.952042 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:17.952111 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:17.952549 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.953154 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.954079 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.954363 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.954462 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:17.955519 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:17.955522 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:17.955524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:17.955545 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:17.956911 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.957349 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.957352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.957354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.957435 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:17.957762 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:17.957765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:17.957767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:17.958035 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:17.958174 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:17.958480 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 18:40:17.958582 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:40:17.958833 21328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:17.951209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36675 (local address 127.8.241.194:59055)
0806 18:40:17.951517 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:17.951532 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:17.951637 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:17.952146 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:17.952149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:17.952334 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:17.952434 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:17.952449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.953304 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.953308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:17.953964 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:17.953974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:17.954458 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:17.954461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:17.954526 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:17.954979 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:17.954994 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:17.955394 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:17.957052 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:17.957056 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:17.957058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:17.957250 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:17.957529 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:17.957532 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:17.957534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:17.957612 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:17.958223 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:17.958227 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:17.958318 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 18:40:17.958407 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 18:40:17.958559 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for oryx/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab.
Entry for principal oryx/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab.
I20260806 18:40:17.978235 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:36675
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab
--principal=oryx/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:18.092157 21332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:18.092394 21332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:18.092435 21332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:18.096374 21332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:18.096426 21332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:18.096462 21332 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:40:18.096498 21332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:18.101523 21332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab
--principal=oryx/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.241.254:36675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:18.102445 21332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:18.103401 21332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:40:18.112810 21332 server_base.cc:1034] running on GCE node
W20260806 18:40:18.112741 21347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:18.112747 21339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:18.112741 21345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:18.113405 21332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:18.114015 21332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:18.115167 21332 hybrid_clock.cc:648] HybridClock initialized: now 1786041618115149 us; error 29 us; skew 500 ppm
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041618, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:18.117724 21332 init.cc:377] Logged in from keytab as oryx/127.8.241.195@KRBTEST.COM (short username oryx)
I20260806 18:40:18.118582 21332 webserver.cc:460] Webserver started at http://127.8.241.195:45257/ using document root <none> and password file <none>
I20260806 18:40:18.119127 21332 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:18.119192 21332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:18.119395 21332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:18.121500 21332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "4ac77e90073143b0a6bf8ccac27d03c7"
format_stamp: "Formatted at 2026-08-06 18:40:18 on dist-test-slave-k5rr"
server_key: "bd39aa1d0c0b6e96f36c660b4ffb5779"
server_key_iv: "f235bcc581bb148dd78ce329dfda49cd"
server_key_version: "encryptionkey@0"
I20260806 18:40:18.122067 21332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "4ac77e90073143b0a6bf8ccac27d03c7"
format_stamp: "Formatted at 2026-08-06 18:40:18 on dist-test-slave-k5rr"
server_key: "bd39aa1d0c0b6e96f36c660b4ffb5779"
server_key_iv: "f235bcc581bb148dd78ce329dfda49cd"
server_key_version: "encryptionkey@0"
I20260806 18:40:18.125734 21332 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:40:18.128037 21358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:18.129287 21332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 18:40:18.129395 21332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "4ac77e90073143b0a6bf8ccac27d03c7"
format_stamp: "Formatted at 2026-08-06 18:40:18 on dist-test-slave-k5rr"
server_key: "bd39aa1d0c0b6e96f36c660b4ffb5779"
server_key_iv: "f235bcc581bb148dd78ce329dfda49cd"
server_key_version: "encryptionkey@0"
I20260806 18:40:18.129495 21332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:18.160137 21332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:18.163030 21332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:18.163216 21332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:18.164742 21332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:18.164794 21332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:18.164851 21332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:18.164880 21332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041618, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:18.175534 21332 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:33467
I20260806 18:40:18.175582 21521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:33467 every 8 connection(s)
I20260806 18:40:18.176671 21332 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:18.178045 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.165519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:51045 (local address 127.8.241.254:36675)
0806 18:40:18.165632 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.165636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.166713 (+ 1077us) server_negotiation.cc:407] Connection header received
0806 18:40:18.167718 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.167722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.167779 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.167833 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:18.169311 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.169869 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.171116 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.171376 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.171490 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.174353 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:18.174358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:18.174360 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:18.174392 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:18.175881 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.176420 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.176422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.176424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.176498 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.176852 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.176855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.176856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.177094 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:18.177194 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.177822 (+ 628us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.177924 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:40:18.178893 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.166083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36675 (local address 127.8.241.195:51045)
0806 18:40:18.166566 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.166608 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.167482 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.168260 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.168267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.168598 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:18.169159 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.169182 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.170004 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.170009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.170987 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.170998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.171548 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.171553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.171755 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.172406 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:18.172430 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:18.174159 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:18.175991 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:18.175996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:18.176010 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:18.176333 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:18.176575 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:18.176578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:18.176580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:18.176751 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:18.177246 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:18.177252 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:18.177546 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.177759 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.178072 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":136,"threads_started":1}
I20260806 18:40:18.180008 21524 heartbeater.cc:347] Connected to a master server at 127.8.241.254:36675
I20260806 18:40:18.180150 21524 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:18.180727 21524 heartbeater.cc:511] Master 127.8.241.254:36675 requested a full tablet report, sending...
I20260806 18:40:18.181951 20809 ts_manager.cc:194] Registered new tserver with Master: 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467)
I20260806 18:40:18.182657 20809 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:51045
I20260806 18:40:18.184926 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 21332
I20260806 18:40:18.185094 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:18.185381 9159 external_mini_cluster.cc:1428] Setting key 97138037262144bcd9464c2165d17d53
I20260806 18:40:18.187525 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:40:18.189379 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.182375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:45333 (local address 127.8.241.254:36675)
0806 18:40:18.182484 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.182488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.182664 (+ 176us) server_negotiation.cc:407] Connection header received
0806 18:40:18.182765 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.182768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.182810 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.182872 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:18.183448 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.183979 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.185055 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.185315 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.185403 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.186560 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:18.186564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:18.186567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:18.186593 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:18.187928 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.188372 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.188374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.188375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.188440 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.188691 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.188692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.188693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.188843 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:18.188912 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.189178 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.189267 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:40:18.189478 21532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.182308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36675 (local address 127.8.241.195:45333)
0806 18:40:18.182590 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.182601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.182691 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.183018 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.183020 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.183205 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:18.183320 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.183328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.184100 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.184104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.184925 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.184935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.185415 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.185418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.185521 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.185990 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:18.186005 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:18.186415 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:18.188035 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:18.188039 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:18.188041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:18.188289 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:18.188522 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:18.188525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:18.188527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:18.188619 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:18.188920 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:18.188923 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:18.189039 (+ 116us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.189139 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.189278 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":117,"threads_started":1}
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041618, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:40:18.212261 20882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:18.206514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52488 (local address 127.8.241.254:36675)
0806 18:40:18.206641 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.206645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.206658 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:18.206703 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.206706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.206746 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.206814 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:18.207496 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.207991 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.208728 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.208925 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.208990 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.212101 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52488: BlockingRecv error: recv got EOF from 127.0.0.1:52488 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041618, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:18.221107 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.213702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52498 (local address 127.8.241.254:36675)
0806 18:40:18.213827 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.213832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.213953 (+ 121us) server_negotiation.cc:407] Connection header received
0806 18:40:18.214094 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.214097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.214135 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.214196 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:18.214775 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.215263 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.215999 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.216209 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.216294 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.218459 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:18.218463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:18.218465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:18.218489 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:18.219562 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.219945 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.219949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.219951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.220027 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.220269 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.220272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.220274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.220458 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:18.220576 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.220804 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.220914 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:40:18.223941 20809 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:40:18.227133 20809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:18.241972 21560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.235565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:33467 (local address 127.0.0.1:37534)
0806 18:40:18.236480 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.236530 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.236677 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.237531 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.237537 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.237559 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:18.237686 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.237694 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.239079 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.239082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.240377 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.240389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.241406 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.241410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.241524 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.241569 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.241682 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.241811 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":18,"thread_start_us":87,"threads_started":1}
I20260806 18:40:18.241964 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.235707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.8.241.195:33467)
0806 18:40:18.236183 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.236196 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.236633 (+ 437us) server_negotiation.cc:407] Connection header received
0806 18:40:18.237134 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.237140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.237351 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.237523 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:18.237799 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.238965 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.240500 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.241266 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.241373 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.241551 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.241730 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.241812 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":142,"threads_started":1}
I20260806 18:40:18.242483 21554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.235831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:42267 (local address 127.0.0.1:38666)
0806 18:40:18.236835 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.236848 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.236933 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.237672 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.237676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.237692 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:18.237871 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.237879 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.239225 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.239229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.240281 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.240291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.242137 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.242139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.242212 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.242229 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.242289 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.242354 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":63,"spinlock_wait_cycles":2432,"thread_start_us":120,"threads_started":1}
I20260806 18:40:18.242599 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.236048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38666 (local address 127.8.241.194:42267)
0806 18:40:18.236409 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.236421 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.237088 (+ 667us) server_negotiation.cc:407] Connection header received
0806 18:40:18.237162 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.237166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.237379 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.237523 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:18.238014 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.239128 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.240401 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.241274 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.241374 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.241551 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.242352 (+ 801us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.242460 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":132,"threads_started":1}
I20260806 18:40:18.243429 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.236325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34265 (local address 127.0.0.1:45198)
0806 18:40:18.236895 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.236907 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.237000 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.237705 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.237708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.237724 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:18.237859 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.237864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.238874 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.238877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.240377 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.240389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.241932 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.241935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.243152 (+ 1217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.243175 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.243241 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.243310 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":114,"thread_start_us":95,"threads_started":1}
I20260806 18:40:18.243829 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.236542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.8.241.193:34265)
0806 18:40:18.236909 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.236916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.236976 (+ 60us) server_negotiation.cc:407] Connection header received
0806 18:40:18.237176 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.237182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.237376 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.237537 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:18.237981 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.238767 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.241093 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.241809 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.241912 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.242056 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.243648 (+ 1592us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.243729 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":83,"threads_started":1}
I20260806 18:40:18.245434 21429 tablet_service.cc:1467] Processing CreateTablet for tablet 75bb565b419b4c8796bde720f0499ce0 (DEFAULT_TABLE table=test-table [id=046a9a68d65144479151177b7a0ec72a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:18.245433 21223 tablet_service.cc:1467] Processing CreateTablet for tablet 75bb565b419b4c8796bde720f0499ce0 (DEFAULT_TABLE table=test-table [id=046a9a68d65144479151177b7a0ec72a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:18.246143 21016 tablet_service.cc:1467] Processing CreateTablet for tablet 75bb565b419b4c8796bde720f0499ce0 (DEFAULT_TABLE table=test-table [id=046a9a68d65144479151177b7a0ec72a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:18.246654 21429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75bb565b419b4c8796bde720f0499ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.246649 21223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75bb565b419b4c8796bde720f0499ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.247043 21016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75bb565b419b4c8796bde720f0499ce0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.253207 21565 tablet_bootstrap.cc:492] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: Bootstrap starting.
I20260806 18:40:18.254803 21566 tablet_bootstrap.cc:492] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: Bootstrap starting.
I20260806 18:40:18.255714 21565 tablet_bootstrap.cc:654] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.256399 21568 tablet_bootstrap.cc:492] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: Bootstrap starting.
I20260806 18:40:18.256644 21565 log.cc:824] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:18.257122 21566 tablet_bootstrap.cc:654] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.257925 21566 log.cc:824] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:18.258615 21568 tablet_bootstrap.cc:654] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.258966 21565 tablet_bootstrap.cc:492] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: No bootstrap required, opened a new log
I20260806 18:40:18.259169 21565 ts_tablet_manager.cc:1397] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:40:18.259423 21568 log.cc:824] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:18.260039 21566 tablet_bootstrap.cc:492] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: No bootstrap required, opened a new log
I20260806 18:40:18.260228 21566 ts_tablet_manager.cc:1397] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 18:40:18.262333 21568 tablet_bootstrap.cc:492] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: No bootstrap required, opened a new log
I20260806 18:40:18.262522 21568 ts_tablet_manager.cc:1397] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:40:18.263840 21565 raft_consensus.cc:348] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } }
I20260806 18:40:18.263952 21566 raft_consensus.cc:348] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } }
I20260806 18:40:18.264134 21565 raft_consensus.cc:374] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.264146 21566 raft_consensus.cc:374] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.264186 21565 raft_consensus.cc:729] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50a858476fb4c17a9d15a34b0d7028a, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.264186 21566 raft_consensus.cc:729] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac77e90073143b0a6bf8ccac27d03c7, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.264694 21566 consensus_queue.cc:260] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } }
I20260806 18:40:18.264694 21565 consensus_queue.cc:260] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } }
I20260806 18:40:18.265743 21565 ts_tablet_manager.cc:1428] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260806 18:40:18.265743 21566 ts_tablet_manager.cc:1428] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:40:18.265846 21568 raft_consensus.cc:348] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } }
I20260806 18:40:18.266095 21568 raft_consensus.cc:374] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.266136 21524 heartbeater.cc:503] Master 127.8.241.254:36675 was elected leader, sending a full tablet report...
I20260806 18:40:18.266147 21568 raft_consensus.cc:729] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b18e54b70a4fdda56847238660aabc, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.266145 21116 heartbeater.cc:503] Master 127.8.241.254:36675 was elected leader, sending a full tablet report...
I20260806 18:40:18.266407 21576 raft_consensus.cc:482] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:18.266559 21576 raft_consensus.cc:504] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } }
I20260806 18:40:18.266644 21568 consensus_queue.cc:260] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } }
I20260806 18:40:18.267702 21319 heartbeater.cc:503] Master 127.8.241.254:36675 was elected leader, sending a full tablet report...
I20260806 18:40:18.268251 21568 ts_tablet_manager.cc:1428] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:40:18.269528 21576 leader_election.cc:288] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467), f2b18e54b70a4fdda56847238660aabc (127.8.241.194:42267)
I20260806 18:40:18.274600 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.269599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:57401 (local address 127.8.241.194:42267)
0806 18:40:18.269720 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.269724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.270115 (+ 391us) server_negotiation.cc:407] Connection header received
0806 18:40:18.270311 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.270314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.270359 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.270406 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:18.270941 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.271744 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.273348 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.273965 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.274088 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.274140 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.274413 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.274483 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:40:18.274600 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.269225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:41047 (local address 127.8.241.195:33467)
0806 18:40:18.269347 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.269352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.270264 (+ 912us) server_negotiation.cc:407] Connection header received
0806 18:40:18.270420 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.270423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.270464 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.270507 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:18.270951 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.271744 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.273344 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.273973 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.274088 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.274140 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.274412 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.274480 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:40:18.274694 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.269208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:42267 (local address 127.8.241.193:57401)
0806 18:40:18.270030 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.270044 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.270188 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.270669 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.270671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.270708 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:18.270817 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.270825 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.272109 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.272112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.273253 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.273261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.274084 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.274088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.274242 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.274317 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.274393 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.274482 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":49,"spinlock_wait_cycles":2688,"thread_start_us":101,"threads_started":1}
I20260806 18:40:18.274777 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.269260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:33467 (local address 127.8.241.193:41047)
0806 18:40:18.270192 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.270204 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.270318 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.270636 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.270638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.270658 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:18.270823 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.270829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.271888 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.271891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.273220 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.273232 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.274084 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.274088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.274261 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.274322 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.274393 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.274482 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":87,"spinlock_wait_cycles":9088,"thread_start_us":112,"threads_started":1}
I20260806 18:40:18.275264 21256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bb565b419b4c8796bde720f0499ce0" candidate_uuid: "b50a858476fb4c17a9d15a34b0d7028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b18e54b70a4fdda56847238660aabc" is_pre_election: true
I20260806 18:40:18.275283 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bb565b419b4c8796bde720f0499ce0" candidate_uuid: "b50a858476fb4c17a9d15a34b0d7028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" is_pre_election: true
I20260806 18:40:18.275528 21256 raft_consensus.cc:2454] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50a858476fb4c17a9d15a34b0d7028a in term 0.
I20260806 18:40:18.275533 21461 raft_consensus.cc:2454] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50a858476fb4c17a9d15a34b0d7028a in term 0.
I20260806 18:40:18.275966 20945 leader_election.cc:302] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac77e90073143b0a6bf8ccac27d03c7, b50a858476fb4c17a9d15a34b0d7028a; no voters:
I20260806 18:40:18.276437 21576 raft_consensus.cc:2781] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:18.276527 21576 raft_consensus.cc:482] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:18.276577 21576 raft_consensus.cc:3037] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.277668 21576 raft_consensus.cc:504] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } }
I20260806 18:40:18.277992 21576 leader_election.cc:288] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [CANDIDATE]: Term 1 election: Requested vote from peers 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467), f2b18e54b70a4fdda56847238660aabc (127.8.241.194:42267)
I20260806 18:40:18.278295 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bb565b419b4c8796bde720f0499ce0" candidate_uuid: "b50a858476fb4c17a9d15a34b0d7028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7"
I20260806 18:40:18.278317 21256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bb565b419b4c8796bde720f0499ce0" candidate_uuid: "b50a858476fb4c17a9d15a34b0d7028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b18e54b70a4fdda56847238660aabc"
I20260806 18:40:18.278417 21256 raft_consensus.cc:3037] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.278417 21461 raft_consensus.cc:3037] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.279666 21461 raft_consensus.cc:2454] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50a858476fb4c17a9d15a34b0d7028a in term 1.
I20260806 18:40:18.279666 21256 raft_consensus.cc:2454] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50a858476fb4c17a9d15a34b0d7028a in term 1.
I20260806 18:40:18.280000 20945 leader_election.cc:302] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac77e90073143b0a6bf8ccac27d03c7, b50a858476fb4c17a9d15a34b0d7028a; no voters:
I20260806 18:40:18.280169 21576 raft_consensus.cc:2781] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:18.280282 21576 raft_consensus.cc:686] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 1 LEADER]: Becoming Leader. State: Replica: b50a858476fb4c17a9d15a34b0d7028a, State: Running, Role: LEADER
I20260806 18:40:18.280606 21576 consensus_queue.cc:237] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } }
I20260806 18:40:18.283720 20809 catalog_manager.cc:5495] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a reported cstate change: term changed from 0 to 1, leader changed from <none> to b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193). New cstate: current_term: 1 leader_uuid: "b50a858476fb4c17a9d15a34b0d7028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } health_report { overall_health: UNKNOWN } } }
I20260806 18:40:18.293856 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.290812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45214 (local address 127.8.241.193:34265)
0806 18:40:18.290915 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.290918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.290930 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:40:18.291044 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.291046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.291095 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.291177 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:18.291349 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.291851 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.292614 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.292808 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.292872 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.293116 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:18.293572 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:18.293657 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.293699 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.293750 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 18:40:18.299547 21461 raft_consensus.cc:1261] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Refusing update from remote peer b50a858476fb4c17a9d15a34b0d7028a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:18.299888 21256 raft_consensus.cc:1261] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Refusing update from remote peer b50a858476fb4c17a9d15a34b0d7028a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:18.300215 21578 consensus_queue.cc:1035] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:18.300493 21576 consensus_queue.cc:1035] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:18.308984 21589 mvcc.cc:205] Tried to move back new op lower bound from 7315626468548009984 to 7315626468480692224. Current Snapshot: MvccSnapshot[applied={T|T < 7315626468548009984}]
I20260806 18:40:18.317795 20809 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:18.317970 20809 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:18.320245 20809 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95: Sending DeleteTablet for 3 replicas of tablet 75bb565b419b4c8796bde720f0499ce0
I20260806 18:40:18.321079 21429 tablet_service.cc:1514] Processing DeleteTablet for tablet 75bb565b419b4c8796bde720f0499ce0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:18 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:37534
I20260806 18:40:18.321090 21223 tablet_service.cc:1514] Processing DeleteTablet for tablet 75bb565b419b4c8796bde720f0499ce0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:18 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:38666
I20260806 18:40:18.321683 20809 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:40:18.321771 21016 tablet_service.cc:1514] Processing DeleteTablet for tablet 75bb565b419b4c8796bde720f0499ce0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:18 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:45198
W20260806 18:40:18.322129 20809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:18.322254 21598 tablet_replica.cc:330] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: stopping tablet replica
I20260806 18:40:18.322389 21600 tablet_replica.cc:330] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: stopping tablet replica
I20260806 18:40:18.322593 21599 tablet_replica.cc:330] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: stopping tablet replica
I20260806 18:40:18.322882 21600 raft_consensus.cc:2229] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:40:18.322887 21599 raft_consensus.cc:2229] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:18.323223 21599 raft_consensus.cc:2258] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:18.323484 21600 raft_consensus.cc:2258] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:18.323671 21598 raft_consensus.cc:2229] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:18.323979 21598 raft_consensus.cc:2258] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:18.325371 21600 ts_tablet_manager.cc:1905] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:18.325459 21599 ts_tablet_manager.cc:1905] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:18.326727 21598 ts_tablet_manager.cc:1905] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:18.327404 21223 tablet_service.cc:1467] Processing CreateTablet for tablet 1ed7a71c3c074b9da5ac679939702555 (DEFAULT_TABLE table=test-table [id=e2e25dbc26da4b3289f18c96cfb08ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:18.327189 21016 tablet_service.cc:1467] Processing CreateTablet for tablet 1ed7a71c3c074b9da5ac679939702555 (DEFAULT_TABLE table=test-table [id=e2e25dbc26da4b3289f18c96cfb08ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:18.327706 21223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ed7a71c3c074b9da5ac679939702555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.327766 21016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ed7a71c3c074b9da5ac679939702555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.328912 21599 ts_tablet_manager.cc:1918] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:18.329044 21599 log.cc:1196] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/75bb565b419b4c8796bde720f0499ce0
I20260806 18:40:18.329348 21599 ts_tablet_manager.cc:1939] T 75bb565b419b4c8796bde720f0499ce0 P f2b18e54b70a4fdda56847238660aabc: Deleting consensus metadata
I20260806 18:40:18.330233 21565 tablet_bootstrap.cc:492] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: Bootstrap starting.
I20260806 18:40:18.330264 21568 tablet_bootstrap.cc:492] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: Bootstrap starting.
I20260806 18:40:18.330602 20784 catalog_manager.cc:4841] TS f2b18e54b70a4fdda56847238660aabc (127.8.241.194:42267): tablet 75bb565b419b4c8796bde720f0499ce0 (table test-table [id=046a9a68d65144479151177b7a0ec72a]) successfully deleted
I20260806 18:40:18.331240 21568 tablet_bootstrap.cc:654] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.331241 21565 tablet_bootstrap.cc:654] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.331403 21429 tablet_service.cc:1467] Processing CreateTablet for tablet 1ed7a71c3c074b9da5ac679939702555 (DEFAULT_TABLE table=test-table [id=e2e25dbc26da4b3289f18c96cfb08ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:18.331692 21429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ed7a71c3c074b9da5ac679939702555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.332778 21565 tablet_bootstrap.cc:492] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: No bootstrap required, opened a new log
I20260806 18:40:18.332878 21565 ts_tablet_manager.cc:1397] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:40:18.333407 21565 raft_consensus.cc:348] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.333542 21565 raft_consensus.cc:374] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.333576 21565 raft_consensus.cc:729] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50a858476fb4c17a9d15a34b0d7028a, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.333698 21565 consensus_queue.cc:260] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.333957 21565 ts_tablet_manager.cc:1428] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260806 18:40:18.334508 21566 tablet_bootstrap.cc:492] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: Bootstrap starting.
I20260806 18:40:18.335484 21566 tablet_bootstrap.cc:654] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.336812 21598 ts_tablet_manager.cc:1918] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:18.338081 21600 ts_tablet_manager.cc:1918] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:18.338172 21568 tablet_bootstrap.cc:492] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: No bootstrap required, opened a new log
I20260806 18:40:18.338282 21568 ts_tablet_manager.cc:1397] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 18:40:18.338325 21566 tablet_bootstrap.cc:492] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: No bootstrap required, opened a new log
I20260806 18:40:18.338291 21598 log.cc:1196] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/75bb565b419b4c8796bde720f0499ce0
I20260806 18:40:18.338421 21566 ts_tablet_manager.cc:1397] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:40:18.338771 21568 raft_consensus.cc:348] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.338896 21568 raft_consensus.cc:374] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.338933 21568 raft_consensus.cc:729] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b18e54b70a4fdda56847238660aabc, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.338908 21566 raft_consensus.cc:348] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.339025 21566 raft_consensus.cc:374] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.339059 21566 raft_consensus.cc:729] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac77e90073143b0a6bf8ccac27d03c7, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.339062 21568 consensus_queue.cc:260] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.339174 21566 consensus_queue.cc:260] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.339326 21568 ts_tablet_manager.cc:1428] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:40:18.339437 21566 ts_tablet_manager.cc:1428] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:18.340240 21598 ts_tablet_manager.cc:1939] T 75bb565b419b4c8796bde720f0499ce0 P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting consensus metadata
I20260806 18:40:18.341085 21600 log.cc:1196] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/75bb565b419b4c8796bde720f0499ce0
I20260806 18:40:18.341413 21600 ts_tablet_manager.cc:1939] T 75bb565b419b4c8796bde720f0499ce0 P b50a858476fb4c17a9d15a34b0d7028a: Deleting consensus metadata
I20260806 18:40:18.342603 20790 catalog_manager.cc:4841] TS b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265): tablet 75bb565b419b4c8796bde720f0499ce0 (table test-table [id=046a9a68d65144479151177b7a0ec72a]) successfully deleted
I20260806 18:40:18.343127 20790 catalog_manager.cc:4841] TS 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467): tablet 75bb565b419b4c8796bde720f0499ce0 (table test-table [id=046a9a68d65144479151177b7a0ec72a]) successfully deleted
I20260806 18:40:18.381529 21580 raft_consensus.cc:482] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:18.381669 21580 raft_consensus.cc:504] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.382774 21580 leader_election.cc:288] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265), 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467)
I20260806 18:40:18.387843 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.382978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34265 (local address 127.8.241.194:35103)
0806 18:40:18.383144 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.383163 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.383271 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.383899 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.383901 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.383916 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:18.384123 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.384135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.385250 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.385253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.386486 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.386499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.387374 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.387378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.387500 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.387547 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.387622 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.387695 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":11}
I20260806 18:40:18.388028 21559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.383348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:51041 (local address 127.8.241.195:33467)
0806 18:40:18.383466 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.383471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.383693 (+ 222us) server_negotiation.cc:407] Connection header received
0806 18:40:18.383901 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.383905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.383961 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.384047 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:18.384416 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.385040 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.386634 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.387219 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.387333 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.387391 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.387805 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.387886 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:40:18.388413 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.383226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:33467 (local address 127.8.241.194:51041)
0806 18:40:18.383599 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.383613 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.383771 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.384137 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.384140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.384161 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:18.384298 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.384305 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.385145 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.385149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.386500 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.386507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.387514 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.387517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.387660 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.387684 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.388163 (+ 479us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.388249 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":39,"thread_start_us":107,"threads_started":1}
I20260806 18:40:18.388602 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.383546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:35103 (local address 127.8.241.193:34265)
0806 18:40:18.383704 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.383709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.383722 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:18.383776 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.383779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.383828 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.383890 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:18.384316 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.385146 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.386633 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.387249 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.387354 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.387400 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.388392 (+ 992us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.388487 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:40:18.388841 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed7a71c3c074b9da5ac679939702555" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" is_pre_election: true
I20260806 18:40:18.388984 21461 raft_consensus.cc:2454] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 0.
I20260806 18:40:18.389159 21045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed7a71c3c074b9da5ac679939702555" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50a858476fb4c17a9d15a34b0d7028a" is_pre_election: true
I20260806 18:40:18.389364 21158 leader_election.cc:302] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac77e90073143b0a6bf8ccac27d03c7, f2b18e54b70a4fdda56847238660aabc; no voters:
I20260806 18:40:18.389433 21045 raft_consensus.cc:2454] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 0.
I20260806 18:40:18.389645 21580 raft_consensus.cc:2781] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:18.389705 21580 raft_consensus.cc:482] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:18.389741 21580 raft_consensus.cc:3037] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.390764 21580 raft_consensus.cc:504] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.391080 21580 leader_election.cc:288] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 election: Requested vote from peers b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265), 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467)
I20260806 18:40:18.391469 21045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed7a71c3c074b9da5ac679939702555" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50a858476fb4c17a9d15a34b0d7028a"
I20260806 18:40:18.391501 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ed7a71c3c074b9da5ac679939702555" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7"
I20260806 18:40:18.391582 21045 raft_consensus.cc:3037] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.391623 21461 raft_consensus.cc:3037] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.392504 21045 raft_consensus.cc:2454] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 1.
I20260806 18:40:18.392750 21461 raft_consensus.cc:2454] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 1.
I20260806 18:40:18.393056 21158 leader_election.cc:302] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac77e90073143b0a6bf8ccac27d03c7, f2b18e54b70a4fdda56847238660aabc; no voters:
I20260806 18:40:18.393230 21580 raft_consensus.cc:2781] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:18.393473 21580 raft_consensus.cc:686] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 1 LEADER]: Becoming Leader. State: Replica: f2b18e54b70a4fdda56847238660aabc, State: Running, Role: LEADER
I20260806 18:40:18.393750 21580 consensus_queue.cc:237] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.395960 20809 catalog_manager.cc:5495] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b18e54b70a4fdda56847238660aabc (127.8.241.194). New cstate: current_term: 1 leader_uuid: "f2b18e54b70a4fdda56847238660aabc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } health_report { overall_health: UNKNOWN } } }
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041617, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.254@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
W20260806 18:40:18.427675 21526 tablet.cc:2369] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:40:18.427776 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.418953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36675 (local address 127.0.0.1:52502)
0806 18:40:18.419229 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.419244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.419349 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.419586 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.419588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.419983 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:18.420078 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.420084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.421091 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.421094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.421681 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.421690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.422264 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.422267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.422348 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.422906 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:18.422918 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:18.424486 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:18.426025 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:18.426036 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:18.426042 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:18.426377 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:18.426638 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:18.426641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:18.426643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:18.426747 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:18.427125 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:18.427132 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:18.427275 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.427407 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.427577 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":113,"threads_started":1}
I20260806 18:40:18.428488 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.419011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52502 (local address 127.8.241.254:36675)
0806 18:40:18.419145 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.419150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.419316 (+ 166us) server_negotiation.cc:407] Connection header received
0806 18:40:18.419422 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.419425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.419474 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.419566 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:18.420200 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.420939 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.421805 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.422146 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.422262 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.424623 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:18.424626 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:18.424627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:18.424649 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:18.425904 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.426475 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.426478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.426479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.426552 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.426830 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.426832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.426833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.427012 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:18.427093 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.427442 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.427652 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":43}
I20260806 18:40:18.432523 20809 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:52502:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:40:18.433408 20809 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:18.438325 21016 tablet_service.cc:1467] Processing CreateTablet for tablet e367189ce63f4076900100c877de3108 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6f873d22b764135a969031b0a3a3f93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:40:18.438395 21429 tablet_service.cc:1467] Processing CreateTablet for tablet e367189ce63f4076900100c877de3108 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6f873d22b764135a969031b0a3a3f93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:40:18.438791 21016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e367189ce63f4076900100c877de3108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.438792 21429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e367189ce63f4076900100c877de3108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.441614 21223 tablet_service.cc:1467] Processing CreateTablet for tablet e367189ce63f4076900100c877de3108 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d6f873d22b764135a969031b0a3a3f93]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:40:18.441797 21565 tablet_bootstrap.cc:492] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: Bootstrap starting.
I20260806 18:40:18.441991 21223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e367189ce63f4076900100c877de3108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.442771 21565 tablet_bootstrap.cc:654] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.444079 21565 tablet_bootstrap.cc:492] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: No bootstrap required, opened a new log
I20260806 18:40:18.444165 21565 ts_tablet_manager.cc:1397] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 18:40:18.444674 21565 raft_consensus.cc:348] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.444774 21568 tablet_bootstrap.cc:492] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: Bootstrap starting.
I20260806 18:40:18.444796 21565 raft_consensus.cc:374] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.444832 21565 raft_consensus.cc:729] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50a858476fb4c17a9d15a34b0d7028a, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.444974 21565 consensus_queue.cc:260] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.445319 21565 ts_tablet_manager.cc:1428] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:18.445380 21576 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.445528 21576 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:40:18.445765 21568 tablet_bootstrap.cc:654] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.446439 21566 tablet_bootstrap.cc:492] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: Bootstrap starting.
W20260806 18:40:18.446612 21320 tablet.cc:2369] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:40:18.447438 21566 tablet_bootstrap.cc:654] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.447876 21568 tablet_bootstrap.cc:492] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: No bootstrap required, opened a new log
I20260806 18:40:18.447969 21568 ts_tablet_manager.cc:1397] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:40:18.448685 21568 raft_consensus.cc:348] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.448875 21568 raft_consensus.cc:374] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.448921 21568 raft_consensus.cc:729] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b18e54b70a4fdda56847238660aabc, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.449162 21566 tablet_bootstrap.cc:492] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: No bootstrap required, opened a new log
I20260806 18:40:18.449143 21568 consensus_queue.cc:260] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.449244 21566 ts_tablet_manager.cc:1397] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:40:18.449462 21580 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.449625 21580 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:40:18.449668 21566 raft_consensus.cc:348] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.449767 21566 raft_consensus.cc:374] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.449795 21566 raft_consensus.cc:729] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac77e90073143b0a6bf8ccac27d03c7, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.449848 21568 ts_tablet_manager.cc:1428] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:18.449906 21566 consensus_queue.cc:260] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.450167 21566 ts_tablet_manager.cc:1428] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:18.450191 21579 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.450392 21579 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 18:40:18.473510 21118 tablet.cc:2369] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:40:18.511479 21576 raft_consensus.cc:482] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:18.511621 21576 raft_consensus.cc:504] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.512086 21576 leader_election.cc:288] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2b18e54b70a4fdda56847238660aabc (127.8.241.194:42267), 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467)
I20260806 18:40:18.512547 21256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e367189ce63f4076900100c877de3108" candidate_uuid: "b50a858476fb4c17a9d15a34b0d7028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b18e54b70a4fdda56847238660aabc" is_pre_election: true
I20260806 18:40:18.512549 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e367189ce63f4076900100c877de3108" candidate_uuid: "b50a858476fb4c17a9d15a34b0d7028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" is_pre_election: true
I20260806 18:40:18.512704 21256 raft_consensus.cc:2454] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50a858476fb4c17a9d15a34b0d7028a in term 0.
I20260806 18:40:18.512727 21461 raft_consensus.cc:2454] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50a858476fb4c17a9d15a34b0d7028a in term 0.
I20260806 18:40:18.513049 20941 leader_election.cc:302] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50a858476fb4c17a9d15a34b0d7028a, f2b18e54b70a4fdda56847238660aabc; no voters:
I20260806 18:40:18.513228 21576 raft_consensus.cc:2781] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:18.513291 21576 raft_consensus.cc:482] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:18.513327 21576 raft_consensus.cc:3037] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.514308 21576 raft_consensus.cc:504] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.514628 21576 leader_election.cc:288] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [CANDIDATE]: Term 1 election: Requested vote from peers f2b18e54b70a4fdda56847238660aabc (127.8.241.194:42267), 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467)
I20260806 18:40:18.514983 21256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e367189ce63f4076900100c877de3108" candidate_uuid: "b50a858476fb4c17a9d15a34b0d7028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2b18e54b70a4fdda56847238660aabc"
I20260806 18:40:18.515024 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e367189ce63f4076900100c877de3108" candidate_uuid: "b50a858476fb4c17a9d15a34b0d7028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7"
I20260806 18:40:18.515098 21256 raft_consensus.cc:3037] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.515128 21461 raft_consensus.cc:3037] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.515981 21461 raft_consensus.cc:2454] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50a858476fb4c17a9d15a34b0d7028a in term 1.
I20260806 18:40:18.516004 21256 raft_consensus.cc:2454] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50a858476fb4c17a9d15a34b0d7028a in term 1.
I20260806 18:40:18.516264 20941 leader_election.cc:302] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4ac77e90073143b0a6bf8ccac27d03c7, b50a858476fb4c17a9d15a34b0d7028a, f2b18e54b70a4fdda56847238660aabc; no voters:
I20260806 18:40:18.516481 21576 raft_consensus.cc:2781] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:18.516557 21576 raft_consensus.cc:686] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [term 1 LEADER]: Becoming Leader. State: Replica: b50a858476fb4c17a9d15a34b0d7028a, State: Running, Role: LEADER
I20260806 18:40:18.516700 21576 consensus_queue.cc:237] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.517414 21578 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: TxnStatusTablet state changed. Reason: New leader b50a858476fb4c17a9d15a34b0d7028a. Latest consensus state: current_term: 1 leader_uuid: "b50a858476fb4c17a9d15a34b0d7028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.517539 21578 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:40:18.517805 21631 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:40:18.517998 21631 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:40:18.518647 20806 catalog_manager.cc:5495] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a reported cstate change: term changed from 0 to 1, leader changed from <none> to b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193). New cstate: current_term: 1 leader_uuid: "b50a858476fb4c17a9d15a34b0d7028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } health_report { overall_health: UNKNOWN } } }
I20260806 18:40:18.548079 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.544752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45228 (local address 127.8.241.193:34265)
0806 18:40:18.544864 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.544868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.544881 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:18.545084 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.545087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.545134 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.545195 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:18.545454 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.545954 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.547114 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.547328 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.547398 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.547705 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:18.547773 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:18.547816 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.547915 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.547965 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 18:40:18.548133 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.544619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34265 (local address 127.0.0.1:45228)
0806 18:40:18.544809 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.544824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.544933 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.545227 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.545230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.545238 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:40:18.545367 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.545373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.546042 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.546046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.547001 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.547012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.547416 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.547420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.547528 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.547585 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:40:18.547835 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:40:18.547847 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.547909 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.547989 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 18:40:18.550860 21256 raft_consensus.cc:1261] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Refusing update from remote peer b50a858476fb4c17a9d15a34b0d7028a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:18.550894 21461 raft_consensus.cc:1261] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Refusing update from remote peer b50a858476fb4c17a9d15a34b0d7028a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:18.551282 21578 consensus_queue.cc:1035] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:18.551471 21576 consensus_queue.cc:1035] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:18.552834 21580 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: TxnStatusTablet state changed. Reason: New leader b50a858476fb4c17a9d15a34b0d7028a. Latest consensus state: current_term: 1 leader_uuid: "b50a858476fb4c17a9d15a34b0d7028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.552958 21580 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:40:18.552888 21579 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: TxnStatusTablet state changed. Reason: New leader b50a858476fb4c17a9d15a34b0d7028a. Latest consensus state: current_term: 1 leader_uuid: "b50a858476fb4c17a9d15a34b0d7028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.552999 21579 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:40:18.554888 21578 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b50a858476fb4c17a9d15a34b0d7028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.554997 21578 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:40:18.556962 21576 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b50a858476fb4c17a9d15a34b0d7028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.557111 21576 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P b50a858476fb4c17a9d15a34b0d7028a: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:40:18.557073 21579 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b50a858476fb4c17a9d15a34b0d7028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.557173 21579 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P 4ac77e90073143b0a6bf8ccac27d03c7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:40:18.558328 21588 mvcc.cc:205] Tried to move back new op lower bound from 7315626469580070912 to 7315626469445853184. Current Snapshot: MvccSnapshot[applied={T|T < 7315626469580070912}]
I20260806 18:40:18.558561 21580 tablet_replica.cc:439] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b50a858476fb4c17a9d15a34b0d7028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } } }
I20260806 18:40:18.558665 21580 tablet_replica.cc:442] T e367189ce63f4076900100c877de3108 P f2b18e54b70a4fdda56847238660aabc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:40:18.565233 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.561362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.8.241.194:42267)
0806 18:40:18.561683 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.561688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.561699 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:40:18.561772 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.561775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.561832 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.561934 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:18.562162 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.562935 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.563664 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.564002 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.564098 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.564276 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:18.564853 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:18.564963 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.565026 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.565083 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260806 18:40:18.575106 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.571947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:39937 (local address 127.8.241.193:34265)
0806 18:40:18.572058 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.572063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.572226 (+ 163us) server_negotiation.cc:407] Connection header received
0806 18:40:18.572355 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.572358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.572408 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.572476 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:18.572724 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.573274 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.574091 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.574322 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.574408 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.574690 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:18.574760 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:18.574816 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.574909 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.574971 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 18:40:18.575122 21639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.571865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34265 (local address 127.8.241.194:39937)
0806 18:40:18.572151 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.572163 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.572259 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.572495 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.572498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.572505 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:40:18.572630 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.572635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.573374 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.573377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.573981 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.573989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.574424 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.574427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.574524 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.574567 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:40:18.574835 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:40:18.574841 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.574899 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.574972 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":110,"threads_started":1}
I20260806 18:40:18.582664 21461 raft_consensus.cc:1261] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Refusing update from remote peer f2b18e54b70a4fdda56847238660aabc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:18.582902 21045 raft_consensus.cc:1261] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Refusing update from remote peer f2b18e54b70a4fdda56847238660aabc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:18.583287 21609 consensus_queue.cc:1035] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:18.583453 21580 consensus_queue.cc:1035] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:18.594592 21586 mvcc.cc:205] Tried to move back new op lower bound from 7315626469710475264 to 7315626468943187968. Current Snapshot: MvccSnapshot[applied={T|T < 7315626469710475264}]
I20260806 18:40:18.606693 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.602982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:40951 (local address 127.8.241.194:42267)
0806 18:40:18.603104 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.603109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.603391 (+ 282us) server_negotiation.cc:407] Connection header received
0806 18:40:18.603529 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.603531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.603573 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.603631 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:18.603861 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.604408 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.605648 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.605930 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.606041 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.606320 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:18.606384 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:18.606435 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.606538 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.606590 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:40:18.607053 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.602911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:42267 (local address 127.8.241.193:40951)
0806 18:40:18.603316 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:18.603331 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:18.603434 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:18.603654 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:18.603657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:18.603666 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:40:18.603777 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.603783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.604493 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.604496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.605496 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:18.605509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.606045 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.606049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:18.606142 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.606199 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:40:18.606464 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:40:18.606470 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:40:18.606533 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:40:18.606620 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":108,"threads_started":1}
I20260806 18:40:18.629956 20806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:18.630137 20806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:18.632447 20806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95: Sending DeleteTablet for 3 replicas of tablet 1ed7a71c3c074b9da5ac679939702555
I20260806 18:40:18.633280 21016 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ed7a71c3c074b9da5ac679939702555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:18 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:45198
I20260806 18:40:18.633361 21223 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ed7a71c3c074b9da5ac679939702555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:18 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:38666
I20260806 18:40:18.633502 21600 tablet_replica.cc:330] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: stopping tablet replica
I20260806 18:40:18.633543 21599 tablet_replica.cc:330] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: stopping tablet replica
I20260806 18:40:18.633628 21600 raft_consensus.cc:2229] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:18.633648 21599 raft_consensus.cc:2229] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:40:18.633754 21600 raft_consensus.cc:2258] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:18.634003 21599 raft_consensus.cc:2258] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:18.634614 21429 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ed7a71c3c074b9da5ac679939702555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:18 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:37534
I20260806 18:40:18.634809 21598 tablet_replica.cc:330] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: stopping tablet replica
I20260806 18:40:18.634931 21598 raft_consensus.cc:2229] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:18.635051 21598 raft_consensus.cc:2258] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:18.635329 21599 ts_tablet_manager.cc:1905] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:18.637849 21598 ts_tablet_manager.cc:1905] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:18.637943 21600 ts_tablet_manager.cc:1905] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:18.638293 21599 ts_tablet_manager.cc:1918] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:40:18.638360 21599 log.cc:1196] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/1ed7a71c3c074b9da5ac679939702555
I20260806 18:40:18.638586 21599 ts_tablet_manager.cc:1939] T 1ed7a71c3c074b9da5ac679939702555 P f2b18e54b70a4fdda56847238660aabc: Deleting consensus metadata
I20260806 18:40:18.639249 20784 catalog_manager.cc:4841] TS f2b18e54b70a4fdda56847238660aabc (127.8.241.194:42267): tablet 1ed7a71c3c074b9da5ac679939702555 (table test-table [id=e2e25dbc26da4b3289f18c96cfb08ef2]) successfully deleted
I20260806 18:40:18.640318 21598 ts_tablet_manager.cc:1918] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:40:18.640372 21598 log.cc:1196] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/1ed7a71c3c074b9da5ac679939702555
I20260806 18:40:18.640542 21598 ts_tablet_manager.cc:1939] T 1ed7a71c3c074b9da5ac679939702555 P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting consensus metadata
I20260806 18:40:18.640944 21600 ts_tablet_manager.cc:1918] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:40:18.641153 20790 catalog_manager.cc:4841] TS 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467): tablet 1ed7a71c3c074b9da5ac679939702555 (table test-table [id=e2e25dbc26da4b3289f18c96cfb08ef2]) successfully deleted
I20260806 18:40:18.641655 21600 log.cc:1196] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/1ed7a71c3c074b9da5ac679939702555
I20260806 18:40:18.641948 21600 ts_tablet_manager.cc:1939] T 1ed7a71c3c074b9da5ac679939702555 P b50a858476fb4c17a9d15a34b0d7028a: Deleting consensus metadata
I20260806 18:40:18.642689 20790 catalog_manager.cc:4841] TS b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265): tablet 1ed7a71c3c074b9da5ac679939702555 (table test-table [id=e2e25dbc26da4b3289f18c96cfb08ef2]) successfully deleted
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:40:18.674010 20882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:18.666343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52504 (local address 127.8.241.254:36675)
0806 18:40:18.666480 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.666485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.666714 (+ 229us) server_negotiation.cc:407] Connection header received
0806 18:40:18.666863 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.666867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.666929 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.667014 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:18.667890 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.668706 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.669891 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.670252 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.670361 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.673863 (+ 3502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52504: BlockingRecv error: recv got EOF from 127.0.0.1:52504 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 18:40:18 dist-test-slave-k5rr krb5kdc[20741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM
I20260806 18:40:18.684473 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.675479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.8.241.254:36675)
0806 18:40:18.675594 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.675599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.675846 (+ 247us) server_negotiation.cc:407] Connection header received
0806 18:40:18.675979 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.675982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.676023 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.676086 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:18.676819 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.677345 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.678422 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.678696 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.678796 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.681457 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:18.681461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:18.681463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:18.681487 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:18.682734 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.683141 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.683144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.683145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.683221 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:18.683491 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:18.683496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:18.683498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:18.683751 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:18.683898 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.684084 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.684262 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:40:18.687247 20806 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:40:18.687711 20806 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:18.692868 21016 tablet_service.cc:1467] Processing CreateTablet for tablet c8c695d3ebb34ab49ca23abcba7cd657 (DEFAULT_TABLE table=test-table [id=202058f2fa3545c0b12c0502ed76d15f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:18.693167 21223 tablet_service.cc:1467] Processing CreateTablet for tablet c8c695d3ebb34ab49ca23abcba7cd657 (DEFAULT_TABLE table=test-table [id=202058f2fa3545c0b12c0502ed76d15f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:18.693250 21429 tablet_service.cc:1467] Processing CreateTablet for tablet c8c695d3ebb34ab49ca23abcba7cd657 (DEFAULT_TABLE table=test-table [id=202058f2fa3545c0b12c0502ed76d15f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:18.693498 21223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8c695d3ebb34ab49ca23abcba7cd657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.693506 21429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8c695d3ebb34ab49ca23abcba7cd657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.695940 21566 tablet_bootstrap.cc:492] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: Bootstrap starting.
I20260806 18:40:18.696197 21016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8c695d3ebb34ab49ca23abcba7cd657. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:18.696995 21566 tablet_bootstrap.cc:654] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.697711 21568 tablet_bootstrap.cc:492] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: Bootstrap starting.
I20260806 18:40:18.698328 21565 tablet_bootstrap.cc:492] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: Bootstrap starting.
I20260806 18:40:18.698688 21568 tablet_bootstrap.cc:654] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.699123 21565 tablet_bootstrap.cc:654] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:18.700189 21568 tablet_bootstrap.cc:492] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: No bootstrap required, opened a new log
I20260806 18:40:18.700279 21568 ts_tablet_manager.cc:1397] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:40:18.700338 21566 tablet_bootstrap.cc:492] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: No bootstrap required, opened a new log
I20260806 18:40:18.700371 21565 tablet_bootstrap.cc:492] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: No bootstrap required, opened a new log
I20260806 18:40:18.700428 21566 ts_tablet_manager.cc:1397] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 18:40:18.700443 21565 ts_tablet_manager.cc:1397] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:40:18.700711 21568 raft_consensus.cc:348] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.700837 21568 raft_consensus.cc:374] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.700870 21568 raft_consensus.cc:729] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b18e54b70a4fdda56847238660aabc, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.700819 21565 raft_consensus.cc:348] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.700914 21565 raft_consensus.cc:374] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.700941 21565 raft_consensus.cc:729] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50a858476fb4c17a9d15a34b0d7028a, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.700897 21566 raft_consensus.cc:348] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.701043 21566 raft_consensus.cc:374] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:18.701085 21566 raft_consensus.cc:729] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac77e90073143b0a6bf8ccac27d03c7, State: Initialized, Role: FOLLOWER
I20260806 18:40:18.701045 21568 consensus_queue.cc:260] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.701124 21565 consensus_queue.cc:260] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.701227 21566 consensus_queue.cc:260] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.701308 21568 ts_tablet_manager.cc:1428] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:18.701360 21565 ts_tablet_manager.cc:1428] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:40:18.701495 21566 ts_tablet_manager.cc:1428] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:40:18.948834 21609 raft_consensus.cc:482] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:18.948997 21609 raft_consensus.cc:504] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.949474 21609 leader_election.cc:288] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265), 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467)
I20260806 18:40:18.950013 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8c695d3ebb34ab49ca23abcba7cd657" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" is_pre_election: true
I20260806 18:40:18.949997 21045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8c695d3ebb34ab49ca23abcba7cd657" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50a858476fb4c17a9d15a34b0d7028a" is_pre_election: true
I20260806 18:40:18.950170 21461 raft_consensus.cc:2454] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 0.
I20260806 18:40:18.950187 21045 raft_consensus.cc:2454] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 0.
I20260806 18:40:18.950482 21158 leader_election.cc:302] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b50a858476fb4c17a9d15a34b0d7028a, f2b18e54b70a4fdda56847238660aabc; no voters:
I20260806 18:40:18.950655 21609 raft_consensus.cc:2781] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:18.950723 21609 raft_consensus.cc:482] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:18.950768 21609 raft_consensus.cc:3037] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.951790 21609 raft_consensus.cc:504] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.952114 21609 leader_election.cc:288] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 election: Requested vote from peers b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265), 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467)
I20260806 18:40:18.952500 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8c695d3ebb34ab49ca23abcba7cd657" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7"
I20260806 18:40:18.952520 21045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8c695d3ebb34ab49ca23abcba7cd657" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50a858476fb4c17a9d15a34b0d7028a"
I20260806 18:40:18.952629 21045 raft_consensus.cc:3037] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.952629 21461 raft_consensus.cc:3037] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:18.953765 21045 raft_consensus.cc:2454] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 1.
I20260806 18:40:18.953765 21461 raft_consensus.cc:2454] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 1.
I20260806 18:40:18.954047 21158 leader_election.cc:302] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac77e90073143b0a6bf8ccac27d03c7, f2b18e54b70a4fdda56847238660aabc; no voters:
I20260806 18:40:18.954293 21609 raft_consensus.cc:2781] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:18.954368 21609 raft_consensus.cc:686] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 1 LEADER]: Becoming Leader. State: Replica: f2b18e54b70a4fdda56847238660aabc, State: Running, Role: LEADER
I20260806 18:40:18.954509 21609 consensus_queue.cc:237] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:18.956133 20807 catalog_manager.cc:5495] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b18e54b70a4fdda56847238660aabc (127.8.241.194). New cstate: current_term: 1 leader_uuid: "f2b18e54b70a4fdda56847238660aabc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } health_report { overall_health: UNKNOWN } } }
I20260806 18:40:18.996129 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:18.993127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.8.241.194:42267)
0806 18:40:18.993254 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:18.993259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:18.993276 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:40:18.993334 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:18.993336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:18.993379 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:18.993442 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:18.993692 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.994191 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.995028 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:18.995240 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:18.995328 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:18.995590 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:18.995677 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:18.995740 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:18.995905 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 18:40:18.995975 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:40:18.998251 21045 raft_consensus.cc:1261] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Refusing update from remote peer f2b18e54b70a4fdda56847238660aabc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:18.998261 21461 raft_consensus.cc:1261] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Refusing update from remote peer f2b18e54b70a4fdda56847238660aabc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:18.998657 21643 consensus_queue.cc:1035] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:18.998835 21609 consensus_queue.cc:1035] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:19.007027 20807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:19.007174 20807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:19.009145 20807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95: Sending DeleteTablet for 3 replicas of tablet c8c695d3ebb34ab49ca23abcba7cd657
I20260806 18:40:19.009789 21223 tablet_service.cc:1514] Processing DeleteTablet for tablet c8c695d3ebb34ab49ca23abcba7cd657 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:19 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:38666
I20260806 18:40:19.009912 21016 tablet_service.cc:1514] Processing DeleteTablet for tablet c8c695d3ebb34ab49ca23abcba7cd657 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:19 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:45198
I20260806 18:40:19.010144 21600 tablet_replica.cc:330] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: stopping tablet replica
I20260806 18:40:19.010165 21429 tablet_service.cc:1514] Processing DeleteTablet for tablet c8c695d3ebb34ab49ca23abcba7cd657 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:19 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:37534
I20260806 18:40:19.010283 21599 tablet_replica.cc:330] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: stopping tablet replica
I20260806 18:40:19.010293 21600 raft_consensus.cc:2229] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:19.010370 21598 tablet_replica.cc:330] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: stopping tablet replica
I20260806 18:40:19.010411 21599 raft_consensus.cc:2229] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:40:19.010452 21600 raft_consensus.cc:2258] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:19.010502 21598 raft_consensus.cc:2229] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:19.010577 21599 raft_consensus.cc:2258] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:19.010632 21598 raft_consensus.cc:2258] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:19.011256 21599 ts_tablet_manager.cc:1905] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:19.011600 21598 ts_tablet_manager.cc:1905] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:19.013631 21599 ts_tablet_manager.cc:1918] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:19.013630 21598 ts_tablet_manager.cc:1918] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:19.013700 21599 log.cc:1196] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/c8c695d3ebb34ab49ca23abcba7cd657
I20260806 18:40:19.013707 21598 log.cc:1196] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/c8c695d3ebb34ab49ca23abcba7cd657
I20260806 18:40:19.013928 21599 ts_tablet_manager.cc:1939] T c8c695d3ebb34ab49ca23abcba7cd657 P f2b18e54b70a4fdda56847238660aabc: Deleting consensus metadata
I20260806 18:40:19.013936 21598 ts_tablet_manager.cc:1939] T c8c695d3ebb34ab49ca23abcba7cd657 P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting consensus metadata
I20260806 18:40:19.014554 20784 catalog_manager.cc:4841] TS f2b18e54b70a4fdda56847238660aabc (127.8.241.194:42267): tablet c8c695d3ebb34ab49ca23abcba7cd657 (table test-table [id=202058f2fa3545c0b12c0502ed76d15f]) successfully deleted
I20260806 18:40:19.014602 20790 catalog_manager.cc:4841] TS 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467): tablet c8c695d3ebb34ab49ca23abcba7cd657 (table test-table [id=202058f2fa3545c0b12c0502ed76d15f]) successfully deleted
I20260806 18:40:19.014869 21600 ts_tablet_manager.cc:1905] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:19.016773 21600 ts_tablet_manager.cc:1918] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:19.016853 21600 log.cc:1196] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/c8c695d3ebb34ab49ca23abcba7cd657
I20260806 18:40:19.017125 21600 ts_tablet_manager.cc:1939] T c8c695d3ebb34ab49ca23abcba7cd657 P b50a858476fb4c17a9d15a34b0d7028a: Deleting consensus metadata
I20260806 18:40:19.017266 20805 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:40:19.017714 20805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:19.017829 20790 catalog_manager.cc:4841] TS b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265): tablet c8c695d3ebb34ab49ca23abcba7cd657 (table test-table [id=202058f2fa3545c0b12c0502ed76d15f]) successfully deleted
I20260806 18:40:19.022276 21016 tablet_service.cc:1467] Processing CreateTablet for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b (DEFAULT_TABLE table=test-table [id=d8c2f99429f04f0d9e8d7bc9f3d6f75e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:19.022476 21223 tablet_service.cc:1467] Processing CreateTablet for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b (DEFAULT_TABLE table=test-table [id=d8c2f99429f04f0d9e8d7bc9f3d6f75e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:19.022473 21429 tablet_service.cc:1467] Processing CreateTablet for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b (DEFAULT_TABLE table=test-table [id=d8c2f99429f04f0d9e8d7bc9f3d6f75e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:19.022581 21016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:19.022732 21223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:19.022732 21429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:19.024514 21565 tablet_bootstrap.cc:492] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a: Bootstrap starting.
I20260806 18:40:19.024935 21568 tablet_bootstrap.cc:492] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: Bootstrap starting.
I20260806 18:40:19.024999 21566 tablet_bootstrap.cc:492] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: Bootstrap starting.
I20260806 18:40:19.025447 21565 tablet_bootstrap.cc:654] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:19.025817 21568 tablet_bootstrap.cc:654] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:19.025921 21566 tablet_bootstrap.cc:654] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:19.026989 21568 tablet_bootstrap.cc:492] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: No bootstrap required, opened a new log
I20260806 18:40:19.027006 21565 tablet_bootstrap.cc:492] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a: No bootstrap required, opened a new log
I20260806 18:40:19.027072 21565 ts_tablet_manager.cc:1397] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:40:19.027070 21568 ts_tablet_manager.cc:1397] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:40:19.027436 21568 raft_consensus.cc:348] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:19.027462 21565 raft_consensus.cc:348] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:19.027536 21568 raft_consensus.cc:374] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:19.027554 21565 raft_consensus.cc:374] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:19.027566 21568 raft_consensus.cc:729] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b18e54b70a4fdda56847238660aabc, State: Initialized, Role: FOLLOWER
I20260806 18:40:19.027581 21565 raft_consensus.cc:729] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50a858476fb4c17a9d15a34b0d7028a, State: Initialized, Role: FOLLOWER
I20260806 18:40:19.027691 21568 consensus_queue.cc:260] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:19.027755 21565 consensus_queue.cc:260] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:19.027977 21568 ts_tablet_manager.cc:1428] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:19.028040 21565 ts_tablet_manager.cc:1428] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:40:19.028950 21566 tablet_bootstrap.cc:492] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: No bootstrap required, opened a new log
I20260806 18:40:19.029047 21566 ts_tablet_manager.cc:1397] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:40:19.029388 21566 raft_consensus.cc:348] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:19.029464 21566 raft_consensus.cc:374] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:19.029484 21566 raft_consensus.cc:729] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac77e90073143b0a6bf8ccac27d03c7, State: Initialized, Role: FOLLOWER
I20260806 18:40:19.029580 21566 consensus_queue.cc:260] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:19.029816 21566 ts_tablet_manager.cc:1428] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:40:19.074450 21643 raft_consensus.cc:482] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:19.074620 21643 raft_consensus.cc:504] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:19.075096 21643 leader_election.cc:288] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265), 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467)
I20260806 18:40:19.075592 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f0fbdfcbe99420b8a70e6dc3b80f79b" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" is_pre_election: true
I20260806 18:40:19.075610 21045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f0fbdfcbe99420b8a70e6dc3b80f79b" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50a858476fb4c17a9d15a34b0d7028a" is_pre_election: true
I20260806 18:40:19.075759 21461 raft_consensus.cc:2454] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 0.
I20260806 18:40:19.075807 21045 raft_consensus.cc:2454] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 0.
I20260806 18:40:19.076078 21158 leader_election.cc:302] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac77e90073143b0a6bf8ccac27d03c7, f2b18e54b70a4fdda56847238660aabc; no voters:
I20260806 18:40:19.076289 21643 raft_consensus.cc:2781] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:19.076357 21643 raft_consensus.cc:482] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:19.076418 21643 raft_consensus.cc:3037] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:19.077478 21643 raft_consensus.cc:504] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:19.077796 21643 leader_election.cc:288] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 election: Requested vote from peers b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265), 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467)
I20260806 18:40:19.078153 21045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f0fbdfcbe99420b8a70e6dc3b80f79b" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50a858476fb4c17a9d15a34b0d7028a"
I20260806 18:40:19.078213 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f0fbdfcbe99420b8a70e6dc3b80f79b" candidate_uuid: "f2b18e54b70a4fdda56847238660aabc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ac77e90073143b0a6bf8ccac27d03c7"
I20260806 18:40:19.078275 21045 raft_consensus.cc:3037] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:19.078339 21461 raft_consensus.cc:3037] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:19.079197 21045 raft_consensus.cc:2454] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 1.
I20260806 18:40:19.079200 21461 raft_consensus.cc:2454] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2b18e54b70a4fdda56847238660aabc in term 1.
I20260806 18:40:19.079483 21158 leader_election.cc:302] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ac77e90073143b0a6bf8ccac27d03c7, f2b18e54b70a4fdda56847238660aabc; no voters:
I20260806 18:40:19.079650 21643 raft_consensus.cc:2781] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:19.079811 21643 raft_consensus.cc:686] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 1 LEADER]: Becoming Leader. State: Replica: f2b18e54b70a4fdda56847238660aabc, State: Running, Role: LEADER
I20260806 18:40:19.079964 21643 consensus_queue.cc:237] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } }
I20260806 18:40:19.081676 20805 catalog_manager.cc:5495] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc reported cstate change: term changed from 0 to 1, leader changed from <none> to f2b18e54b70a4fdda56847238660aabc (127.8.241.194). New cstate: current_term: 1 leader_uuid: "f2b18e54b70a4fdda56847238660aabc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2b18e54b70a4fdda56847238660aabc" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 } health_report { overall_health: UNKNOWN } } }
I20260806 18:40:19.100411 21461 raft_consensus.cc:1261] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Refusing update from remote peer f2b18e54b70a4fdda56847238660aabc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:19.100485 21045 raft_consensus.cc:1261] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Refusing update from remote peer f2b18e54b70a4fdda56847238660aabc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:19.100790 21643 consensus_queue.cc:1035] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50a858476fb4c17a9d15a34b0d7028a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:19.100953 21580 consensus_queue.cc:1035] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ac77e90073143b0a6bf8ccac27d03c7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 33467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:19.130244 20805 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:19.130409 20805 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:19.132246 20805 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c2dd4e5bc874757b5d5667f117b8e95: Sending DeleteTablet for 3 replicas of tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b
I20260806 18:40:19.132764 21016 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:19 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:45198
I20260806 18:40:19.132810 21223 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:19 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:38666
I20260806 18:40:19.132824 21429 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:19 UTC) from {username='oryx', principal='oryx/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:37534
I20260806 18:40:19.133100 21599 tablet_replica.cc:330] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: stopping tablet replica
I20260806 18:40:19.133229 21599 raft_consensus.cc:2229] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:40:19.133432 21599 raft_consensus.cc:2258] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:19.133617 21598 tablet_replica.cc:330] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: stopping tablet replica
I20260806 18:40:19.133729 21598 raft_consensus.cc:2229] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:19.133842 21598 raft_consensus.cc:2258] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:19.133939 21600 tablet_replica.cc:330] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a: stopping tablet replica
I20260806 18:40:19.134069 21600 raft_consensus.cc:2229] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:19.134073 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 20918
I20260806 18:40:19.134184 21600 raft_consensus.cc:2258] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P b50a858476fb4c17a9d15a34b0d7028a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:19.136847 21598 ts_tablet_manager.cc:1905] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:19.139726 21598 ts_tablet_manager.cc:1918] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:40:19.139801 21598 log.cc:1196] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/4f0fbdfcbe99420b8a70e6dc3b80f79b
I20260806 18:40:19.140007 21598 ts_tablet_manager.cc:1939] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P 4ac77e90073143b0a6bf8ccac27d03c7: Deleting consensus metadata
I20260806 18:40:19.141809 20790 catalog_manager.cc:4841] TS 4ac77e90073143b0a6bf8ccac27d03c7 (127.8.241.195:33467): tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b (table test-table [id=d8c2f99429f04f0d9e8d7bc9f3d6f75e]) successfully deleted
I20260806 18:40:19.143131 21599 ts_tablet_manager.cc:1905] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:19.145180 21599 ts_tablet_manager.cc:1918] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:40:19.145256 21599 log.cc:1196] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/4f0fbdfcbe99420b8a70e6dc3b80f79b
I20260806 18:40:19.145483 21599 ts_tablet_manager.cc:1939] T 4f0fbdfcbe99420b8a70e6dc3b80f79b P f2b18e54b70a4fdda56847238660aabc: Deleting consensus metadata
I20260806 18:40:19.146164 20784 catalog_manager.cc:4841] TS f2b18e54b70a4fdda56847238660aabc (127.8.241.194:42267): tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b (table test-table [id=d8c2f99429f04f0d9e8d7bc9f3d6f75e]) successfully deleted
W20260806 18:40:19.155280 20790 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:34265 (error 108)
I20260806 18:40:19.155519 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 21134
I20260806 18:40:19.158474 21699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:19.158066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34265 (local address 127.0.0.1:45238)
0806 18:40:19.158286 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:19.158358 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:34265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":55,"threads_started":1}
W20260806 18:40:19.159539 20790 catalog_manager.cc:4584] TS b50a858476fb4c17a9d15a34b0d7028a (127.8.241.193:34265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f0fbdfcbe99420b8a70e6dc3b80f79b: Network error: Client connection negotiation failed: client connection to 127.8.241.193:34265: connect: Connection refused (error 111)
I20260806 18:40:19.169998 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 21332
I20260806 18:40:19.181068 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 20760
2026-08-06T18:40:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3229 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:40:19 dist-test-slave-k5rr krb5kdc[21705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:40:19 dist-test-slave-k5rr krb5kdc[21705](info): set up 2 sockets
Aug 06 18:40:19 dist-test-slave-k5rr krb5kdc[21705](info): commencing operation
krb5kdc: starting...
W20260806 18:40:20.318439 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:40:20 dist-test-slave-k5rr krb5kdc[21705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:40:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:40:20.437975 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39873
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:46211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:20.550613 21725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:20.550874 21725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:20.550916 21725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:20.554898 21725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:20.554955 21725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:20.554973 21725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:20.555003 21725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:20.555029 21725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:20.560191 21725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:46211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:20.561475 21725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:20.562598 21725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:20.571208 21734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:20.571218 21732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:20.571543 21725 server_base.cc:1034] running on GCE node
W20260806 18:40:20.571287 21737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:20.572238 21725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:20.573370 21725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:20.574533 21725 hybrid_clock.cc:648] HybridClock initialized: now 1786041620574518 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T18:40:20Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1399 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:40:20 dist-test-slave-k5rr krb5kdc[21746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:40:20 dist-test-slave-k5rr krb5kdc[21746](info): set up 2 sockets
Aug 06 18:40:20 dist-test-slave-k5rr krb5kdc[21746](info): commencing operation
krb5kdc: starting...
W20260806 18:40:21.722879 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:40:21 dist-test-slave-k5rr krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:40:21.843592 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34303
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:34303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:21.956645 21765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:21.956915 21765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:21.956954 21765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:21.960899 21765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:21.960955 21765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:21.960973 21765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:21.960994 21765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:21.961031 21765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:21.966661 21765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:34303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:21.967693 21765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:21.968694 21765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:21.977382 21773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:21.977386 21775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:21.977499 21765 server_base.cc:1034] running on GCE node
W20260806 18:40:21.977382 21772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:21.978307 21765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:21.979447 21765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:21.980611 21765 hybrid_clock.cc:648] HybridClock initialized: now 1786041621980594 us; error 29 us; skew 500 ppm
Aug 06 18:40:21 dist-test-slave-k5rr krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041621, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:21.983333 21765 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:40:21.984278 21765 webserver.cc:460] Webserver started at http://127.8.241.254:33145/ using document root <none> and password file <none>
I20260806 18:40:21.984862 21765 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:21.984933 21765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:21.985204 21765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:21.987440 21765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "d9202fdf8a3b485f85d0286930ded77e"
format_stamp: "Formatted at 2026-08-06 18:40:21 on dist-test-slave-k5rr"
server_key: "ae1f84f9452505741d6417d71fb9c1b1"
server_key_iv: "4865e6f4cfab2b65282f4c9a550c9e9d"
server_key_version: "encryptionkey@0"
I20260806 18:40:21.988005 21765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "d9202fdf8a3b485f85d0286930ded77e"
format_stamp: "Formatted at 2026-08-06 18:40:21 on dist-test-slave-k5rr"
server_key: "ae1f84f9452505741d6417d71fb9c1b1"
server_key_iv: "4865e6f4cfab2b65282f4c9a550c9e9d"
server_key_version: "encryptionkey@0"
I20260806 18:40:21.991992 21765 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:40:21.994642 21783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:21.995890 21765 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:40:21.996033 21765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "d9202fdf8a3b485f85d0286930ded77e"
format_stamp: "Formatted at 2026-08-06 18:40:21 on dist-test-slave-k5rr"
server_key: "ae1f84f9452505741d6417d71fb9c1b1"
server_key_iv: "4865e6f4cfab2b65282f4c9a550c9e9d"
server_key_version: "encryptionkey@0"
I20260806 18:40:21.996140 21765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:22.020885 21765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:22.023764 21765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:22.023967 21765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:22.030979 21765 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:34303
I20260806 18:40:22.030988 21863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:34303 every 8 connection(s)
I20260806 18:40:22.031981 21765 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:22.034806 21864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:22.040673 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 21765
I20260806 18:40:22.040791 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:22.040829 21864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: Bootstrap starting.
I20260806 18:40:22.041142 9159 external_mini_cluster.cc:1428] Setting key 8435aed36f0f2f5e374e3dfd3593eb9b
I20260806 18:40:22.043403 21864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:22.044165 21864 log.cc:824] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:22.046240 21864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: No bootstrap required, opened a new log
Aug 06 18:40:22 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:22.050580 21864 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } }
I20260806 18:40:22.050812 21864 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:22.050870 21864 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9202fdf8a3b485f85d0286930ded77e, State: Initialized, Role: FOLLOWER
I20260806 18:40:22.051375 21864 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } }
I20260806 18:40:22.051548 21864 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:22.051623 21864 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:22.051733 21864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:22.052597 21864 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } }
I20260806 18:40:22.052983 21864 leader_election.cc:302] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9202fdf8a3b485f85d0286930ded77e; no voters:
I20260806 18:40:22.053378 21864 leader_election.cc:288] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:22.053468 21871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:22.053678 21871 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 1 LEADER]: Becoming Leader. State: Replica: d9202fdf8a3b485f85d0286930ded77e, State: Running, Role: LEADER
I20260806 18:40:22.054031 21871 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } }
I20260806 18:40:22.054771 21864 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:22.055745 21873 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9202fdf8a3b485f85d0286930ded77e. Latest consensus state: current_term: 1 leader_uuid: "d9202fdf8a3b485f85d0286930ded77e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } } }
I20260806 18:40:22.055745 21872 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9202fdf8a3b485f85d0286930ded77e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } } }
I20260806 18:40:22.055886 21872 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:22.056095 21868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:22.042551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.8.241.254:34303)
0806 18:40:22.042957 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:22.042977 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:22.043015 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:40:22.043678 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:22.043697 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:22.044045 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:22.044362 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:22.045177 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.046165 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.047298 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.047783 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.047905 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:22.050652 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:22.050658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:22.050673 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:22.050702 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:22.052397 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:22.052824 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:22.052830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:22.052837 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:22.052984 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:22.053255 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:22.053258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:22.053260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:22.053818 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:22.054612 (+ 794us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:22.054768 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 18:40:22.055077 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":104,"threads_started":1}
I20260806 18:40:22.055886 21873 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: This master's current role is: LEADER
W20260806 18:40:22.059404 21894 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:40:22.059480 21894 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:40:22.059551 21884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:22.060426 21884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:22.065866 21884 catalog_manager.cc:1347] Generated new cluster ID: ece5b29d6d234e84bca403d358b5069e
I20260806 18:40:22.065928 21884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:22.091898 21884 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:22.092891 21884 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:22.098477 21884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: Generated new TSK 0
I20260806 18:40:22.099314 21884 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:40:22.136613 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34303
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:22.251681 21906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:22.251960 21906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:22.252020 21906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:22.256119 21906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:22.256184 21906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:22.256268 21906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:22.261590 21906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:22.262673 21906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:22.263722 21906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:22.273334 21911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:22.273367 21914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:22.273434 21906 server_base.cc:1034] running on GCE node
W20260806 18:40:22.273360 21912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:22.273989 21906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:22.274582 21906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:22.275760 21906 hybrid_clock.cc:648] HybridClock initialized: now 1786041622275740 us; error 30 us; skew 500 ppm
Aug 06 18:40:22 dist-test-slave-k5rr krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:22.278539 21906 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:40:22.279389 21906 webserver.cc:460] Webserver started at http://127.8.241.193:38717/ using document root <none> and password file <none>
I20260806 18:40:22.279917 21906 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:22.279985 21906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:22.280161 21906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:22.282214 21906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "c875d32e57b94ab5bbc30df7387df6c5"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "9354e6c929abfe3c2af1226643e22673"
server_key_iv: "28a6d9db679d9b991ad732df02d8e680"
server_key_version: "encryptionkey@0"
I20260806 18:40:22.282728 21906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "c875d32e57b94ab5bbc30df7387df6c5"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "9354e6c929abfe3c2af1226643e22673"
server_key_iv: "28a6d9db679d9b991ad732df02d8e680"
server_key_version: "encryptionkey@0"
I20260806 18:40:22.286438 21906 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:40:22.288734 21922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:22.289955 21906 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:22.290055 21906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "c875d32e57b94ab5bbc30df7387df6c5"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "9354e6c929abfe3c2af1226643e22673"
server_key_iv: "28a6d9db679d9b991ad732df02d8e680"
server_key_version: "encryptionkey@0"
I20260806 18:40:22.290138 21906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:22.319389 21906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:22.322221 21906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:22.322381 21906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:22.322932 21906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:22.323801 21906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:22.323848 21906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:22.323890 21906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:22.323910 21906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:22.332250 21906 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:41983
I20260806 18:40:22.332278 22080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:41983 every 8 connection(s)
I20260806 18:40:22.333165 21906 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:22.333724 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 21906
I20260806 18:40:22.333854 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:22.334147 9159 external_mini_cluster.cc:1428] Setting key b97ecce30381d41600db084c69c80c59
Aug 06 18:40:22 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:40:22.346565 21868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:22.334955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:49753 (local address 127.8.241.254:34303)
0806 18:40:22.335083 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:22.335088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:22.336352 (+ 1264us) server_negotiation.cc:407] Connection header received
0806 18:40:22.337261 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:22.337264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:22.337319 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:22.337391 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:22.338595 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.339132 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.340268 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.340509 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.340601 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:22.343027 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:22.343031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:22.343034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:22.343060 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:22.344317 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:22.344858 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:22.344862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:22.344864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:22.344944 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:22.345284 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:22.345288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:22.345290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:22.345535 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:22.345633 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:22.346241 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 18:40:22.346375 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:40:22.347362 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:22.335430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:49753)
0806 18:40:22.336187 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:22.336241 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:22.337056 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:22.337574 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:22.337582 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:22.337934 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:22.338426 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:22.338439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.339259 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.339262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:22.340141 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:22.340152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.340657 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.340662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:22.340827 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:22.341433 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:22.341451 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:22.342863 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:22.344460 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:22.344466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:22.344482 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:22.344759 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:22.345049 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:22.345052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:22.345055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:22.345191 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:22.345645 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:22.345653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:22.345952 (+ 299us) client_negotiation.cc:769] Sending connection context
0806 18:40:22.346167 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 18:40:22.346508 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":120,"threads_started":1}
I20260806 18:40:22.348560 22081 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34303
I20260806 18:40:22.348836 22081 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:22.349473 22081 heartbeater.cc:511] Master 127.8.241.254:34303 requested a full tablet report, sending...
I20260806 18:40:22.351064 21806 ts_manager.cc:194] Registered new tserver with Master: c875d32e57b94ab5bbc30df7387df6c5 (127.8.241.193:41983)
I20260806 18:40:22.352183 21806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:49753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:40:22.362257 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34303
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:22.476025 22088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:22.476295 22088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:22.476342 22088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:22.480430 22088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:22.480495 22088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:22.480559 22088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:22.486178 22088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:22.487322 22088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:22.488385 22088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:22.498276 22095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:22.498389 22088 server_base.cc:1034] running on GCE node
W20260806 18:40:22.498274 22096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:22.498281 22099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:22.498958 22088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:22.499634 22088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:22.500810 22088 hybrid_clock.cc:648] HybridClock initialized: now 1786041622500775 us; error 44 us; skew 500 ppm
Aug 06 18:40:22 dist-test-slave-k5rr krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:22.503509 22088 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:40:22.504408 22088 webserver.cc:460] Webserver started at http://127.8.241.194:42073/ using document root <none> and password file <none>
I20260806 18:40:22.504976 22088 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:22.505076 22088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:22.505273 22088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:22.507339 22088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "b2f72aa417be478d894b811b7a8cf896"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "2361eb887ee0d7aa68e8032958567f73"
server_key_iv: "81abf0139c86618a1921bce2b013508b"
server_key_version: "encryptionkey@0"
I20260806 18:40:22.507862 22088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "b2f72aa417be478d894b811b7a8cf896"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "2361eb887ee0d7aa68e8032958567f73"
server_key_iv: "81abf0139c86618a1921bce2b013508b"
server_key_version: "encryptionkey@0"
I20260806 18:40:22.511608 22088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:40:22.513993 22107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:22.515154 22088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:22.515265 22088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "b2f72aa417be478d894b811b7a8cf896"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "2361eb887ee0d7aa68e8032958567f73"
server_key_iv: "81abf0139c86618a1921bce2b013508b"
server_key_version: "encryptionkey@0"
I20260806 18:40:22.515357 22088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:22.546456 22088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:22.549831 22088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:22.550021 22088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:22.550613 22088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:22.551493 22088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:22.551540 22088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:22.551595 22088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:22.551620 22088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:22.561976 22088 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:34413
I20260806 18:40:22.562011 22278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:34413 every 8 connection(s)
I20260806 18:40:22.562850 22088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:22.569080 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 22088
I20260806 18:40:22.569203 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:40:22.569478 9159 external_mini_cluster.cc:1428] Setting key 094bc1a254cafd8042c22903727c5559
Aug 06 18:40:22 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:22.576442 21868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:22.564583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:45351 (local address 127.8.241.254:34303)
0806 18:40:22.564699 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:22.564703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:22.565568 (+ 865us) server_negotiation.cc:407] Connection header received
0806 18:40:22.566931 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:22.566934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:22.566995 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:22.567072 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:22.568279 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.568795 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.570070 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.570351 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.570465 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:22.572956 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:22.572959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:22.572961 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:22.572984 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:22.574247 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:22.574830 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:22.574833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:22.574834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:22.574915 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:22.575274 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:22.575276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:22.575277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:22.575459 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:22.575530 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:22.576185 (+ 655us) server_negotiation.cc:299] Negotiation successful
0806 18:40:22.576311 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":38}
I20260806 18:40:22.577643 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:22.564935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:45351)
0806 18:40:22.565422 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:22.565465 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:22.566717 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:22.567266 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:22.567273 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:22.567640 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:22.568136 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:22.568150 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.568915 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.568919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:22.569941 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:22.569955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.570494 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.570497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:22.570638 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:22.571226 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:22.571243 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:22.572792 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:22.574362 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:22.574368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:22.574396 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:22.574712 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:22.575012 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:22.575015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:22.575017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:22.575191 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:22.575550 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:22.575556 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:22.575891 (+ 335us) client_negotiation.cc:769] Sending connection context
0806 18:40:22.576126 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 18:40:22.576500 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":107,"threads_started":1}
I20260806 18:40:22.578788 22279 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34303
I20260806 18:40:22.579064 22279 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:22.579680 22279 heartbeater.cc:511] Master 127.8.241.254:34303 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:40:22.580785 21806 ts_manager.cc:194] Registered new tserver with Master: b2f72aa417be478d894b811b7a8cf896 (127.8.241.194:34413)
I20260806 18:40:22.581404 21806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:45351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:40:22.597339 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34303
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:22.712634 22286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:22.712888 22286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:22.712929 22286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:22.716921 22286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:22.716989 22286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:22.717064 22286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:22.722215 22286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:22.723237 22286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:22.724263 22286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:22.733989 22292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:22.733989 22296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:22.734046 22291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:22.734251 22286 server_base.cc:1034] running on GCE node
I20260806 18:40:22.734673 22286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:22.735277 22286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:22.736430 22286 hybrid_clock.cc:648] HybridClock initialized: now 1786041622736404 us; error 46 us; skew 500 ppm
Aug 06 18:40:22 dist-test-slave-k5rr krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:22.739032 22286 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:40:22.739887 22286 webserver.cc:460] Webserver started at http://127.8.241.195:40143/ using document root <none> and password file <none>
I20260806 18:40:22.740425 22286 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:22.740512 22286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:22.740797 22286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:22.742885 22286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "0415179bd79a4fdbb82fac5aa313550a"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "0851377d4b8a5b68199faea239c239aa"
server_key_iv: "d949f74035acb15b14b5ae79a603c23c"
server_key_version: "encryptionkey@0"
I20260806 18:40:22.743417 22286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "0415179bd79a4fdbb82fac5aa313550a"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "0851377d4b8a5b68199faea239c239aa"
server_key_iv: "d949f74035acb15b14b5ae79a603c23c"
server_key_version: "encryptionkey@0"
I20260806 18:40:22.747115 22286 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:40:22.749488 22314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:22.750710 22286 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:22.750820 22286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "0415179bd79a4fdbb82fac5aa313550a"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "0851377d4b8a5b68199faea239c239aa"
server_key_iv: "d949f74035acb15b14b5ae79a603c23c"
server_key_version: "encryptionkey@0"
I20260806 18:40:22.750913 22286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:22.782588 22286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:22.785382 22286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:22.785573 22286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:22.786172 22286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:22.787031 22286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:22.787081 22286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:22.787135 22286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:22.787165 22286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:22.796070 22286 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:45381
I20260806 18:40:22.796087 22473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:45381 every 8 connection(s)
I20260806 18:40:22.797034 22286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:22.804625 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 22286
I20260806 18:40:22.804782 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:22.805146 9159 external_mini_cluster.cc:1428] Setting key 227b1d5761a0714233b5848813e81380
Aug 06 18:40:22 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:22.810122 21868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:22.798802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:44195 (local address 127.8.241.254:34303)
0806 18:40:22.798921 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:22.798925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:22.799972 (+ 1047us) server_negotiation.cc:407] Connection header received
0806 18:40:22.800899 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:22.800903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:22.800973 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:22.801064 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:22.802237 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.802764 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.803893 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.804122 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.804213 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:22.807030 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:22.807033 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:22.807035 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:22.807061 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:22.808323 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:22.808810 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:22.808813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:22.808814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:22.808889 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:22.809196 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:22.809198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:22.809199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:22.809375 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:22.809453 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:22.809894 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 18:40:22.810001 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:40:22.811010 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:22.799152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:44195)
0806 18:40:22.799795 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:22.799835 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:22.800692 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:22.801249 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:22.801257 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:22.801615 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:22.802090 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:22.802103 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.802882 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.802885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:22.803764 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:22.803774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:22.804264 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:22.804268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:22.804471 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:22.805142 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:22.805163 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:22.806828 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:22.808444 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:22.808449 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:22.808461 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:22.808735 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:22.808957 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:22.808959 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:22.808961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:22.809123 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:22.809467 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:22.809472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:22.809708 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 18:40:22.809863 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 18:40:22.810127 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":106,"threads_started":1}
I20260806 18:40:22.812124 22476 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34303
I20260806 18:40:22.812346 22476 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:22.812829 22476 heartbeater.cc:511] Master 127.8.241.254:34303 requested a full tablet report, sending...
I20260806 18:40:22.813859 21806 ts_manager.cc:194] Registered new tserver with Master: 0415179bd79a4fdbb82fac5aa313550a (127.8.241.195:45381)
I20260806 18:40:22.814424 21806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:44195
I20260806 18:40:22.819206 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab.
Entry for principal oryx/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab.
WARNING: no policy specified for oryx/127.8.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab.
Entry for principal oryx/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab.
WARNING: no policy specified for oryx/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab.
Entry for principal oryx/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab.
I20260806 18:40:22.888602 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 21906
I20260806 18:40:22.901672 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 22088
I20260806 18:40:22.915899 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 22286
I20260806 18:40:22.935681 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 21765
I20260806 18:40:22.948223 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34303
--webserver_interface=127.8.241.254
--webserver_port=33145
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:34303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:23.059597 22487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:23.059866 22487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:23.059913 22487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:23.063869 22487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:23.063927 22487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:23.063951 22487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:23.063977 22487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:23.064004 22487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:23.069566 22487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:34303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=33145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:23.070613 22487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:23.071650 22487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:23.080219 22496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:23.080206 22493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:23.080258 22487 server_base.cc:1034] running on GCE node
W20260806 18:40:23.080209 22500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:23.081069 22487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:23.082151 22487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:23.083321 22487 hybrid_clock.cc:648] HybridClock initialized: now 1786041623083299 us; error 33 us; skew 500 ppm
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:23.085995 22487 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:40:23.086887 22487 webserver.cc:460] Webserver started at http://127.8.241.254:33145/ using document root <none> and password file <none>
I20260806 18:40:23.087450 22487 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:23.087534 22487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:23.091408 22487 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:40:23.093538 22509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.094729 22487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:23.094842 22487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "d9202fdf8a3b485f85d0286930ded77e"
format_stamp: "Formatted at 2026-08-06 18:40:21 on dist-test-slave-k5rr"
server_key: "ae1f84f9452505741d6417d71fb9c1b1"
server_key_iv: "4865e6f4cfab2b65282f4c9a550c9e9d"
server_key_version: "encryptionkey@0"
I20260806 18:40:23.095355 22487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:23.116106 22487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:23.118912 22487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:23.119095 22487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:23.126163 22487 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:34303
I20260806 18:40:23.126176 22585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:34303 every 8 connection(s)
I20260806 18:40:23.127247 22487 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:23.132064 22587 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:40:23.134019 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 22487
I20260806 18:40:23.134183 22587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: Bootstrap starting.
I20260806 18:40:23.134538 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:41983
--local_ip_for_outbound_sockets=127.8.241.193
--tserver_master_addrs=127.8.241.254:34303
--webserver_port=38717
--webserver_interface=127.8.241.193
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.241.193
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:40:23.151934 22587 log.cc:824] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:23.158732 22587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:40:23.159081 22587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: Bootstrap complete.
I20260806 18:40:23.163236 22587 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } }
I20260806 18:40:23.163645 22587 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9202fdf8a3b485f85d0286930ded77e, State: Initialized, Role: FOLLOWER
I20260806 18:40:23.164132 22587 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } }
I20260806 18:40:23.164252 22587 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:23.164316 22587 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:23.164412 22587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:40:23.166389 22587 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } }
I20260806 18:40:23.166715 22587 leader_election.cc:302] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9202fdf8a3b485f85d0286930ded77e; no voters:
I20260806 18:40:23.167028 22587 leader_election.cc:288] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:40:23.167166 22594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:40:23.167416 22594 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [term 2 LEADER]: Becoming Leader. State: Replica: d9202fdf8a3b485f85d0286930ded77e, State: Running, Role: LEADER
I20260806 18:40:23.167730 22594 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } }
I20260806 18:40:23.168386 22587 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:23.169275 22596 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9202fdf8a3b485f85d0286930ded77e. Latest consensus state: current_term: 2 leader_uuid: "d9202fdf8a3b485f85d0286930ded77e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } } }
I20260806 18:40:23.169409 22596 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:23.169698 22595 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d9202fdf8a3b485f85d0286930ded77e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9202fdf8a3b485f85d0286930ded77e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34303 } } }
I20260806 18:40:23.169792 22595 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:23.173738 22620 catalog_manager.cc:1259] Loaded cluster ID: ece5b29d6d234e84bca403d358b5069e
I20260806 18:40:23.173797 22620 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: loading cluster ID for follower catalog manager: success
I20260806 18:40:23.176154 22620 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: acquiring CA information for follower catalog manager: success
I20260806 18:40:23.177573 22620 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:40:23.177691 22609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:23.179226 22609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:23.179701 22609 catalog_manager.cc:1259] Loaded cluster ID: ece5b29d6d234e84bca403d358b5069e
I20260806 18:40:23.179757 22609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:23.180411 22609 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:23.180856 22609 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d9202fdf8a3b485f85d0286930ded77e: Loaded TSK: 0
I20260806 18:40:23.181667 22609 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:40:23.253808 22592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:23.254055 22592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:23.254097 22592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:23.258140 22592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:23.258201 22592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:23.258260 22592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:23.263569 22592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.193.keytab
--principal=oryx/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:41983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=38717
--tserver_master_addrs=127.8.241.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:23.264597 22592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:23.265671 22592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:23.276700 22633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:23.276710 22632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:23.276953 22636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:23.276964 22592 server_base.cc:1034] running on GCE node
I20260806 18:40:23.277676 22592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:23.278528 22592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:23.279739 22592 hybrid_clock.cc:648] HybridClock initialized: now 1786041623279702 us; error 43 us; skew 500 ppm
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041623, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:23.283262 22592 init.cc:377] Logged in from keytab as oryx/127.8.241.193@KRBTEST.COM (short username oryx)
I20260806 18:40:23.284416 22592 webserver.cc:460] Webserver started at http://127.8.241.193:38717/ using document root <none> and password file <none>
I20260806 18:40:23.285192 22592 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:23.285312 22592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:23.290203 22592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:40:23.292613 22650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.294044 22592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:40:23.294159 22592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "c875d32e57b94ab5bbc30df7387df6c5"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "9354e6c929abfe3c2af1226643e22673"
server_key_iv: "28a6d9db679d9b991ad732df02d8e680"
server_key_version: "encryptionkey@0"
I20260806 18:40:23.294878 22592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:23.325816 22592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:23.328549 22592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:23.328707 22592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:23.329298 22592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:23.330108 22592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:23.330156 22592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.330197 22592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:23.330217 22592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.338817 22592 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:41983
I20260806 18:40:23.338850 22819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:41983 every 8 connection(s)
I20260806 18:40:23.339820 22592 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:23.342039 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 22592
I20260806 18:40:23.342563 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:34413
--local_ip_for_outbound_sockets=127.8.241.194
--tserver_master_addrs=127.8.241.254:34303
--webserver_port=42073
--webserver_interface=127.8.241.194
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.241.194
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:23.353579 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.342025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:57077)
0806 18:40:23.342851 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:23.342905 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:23.343678 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:23.344821 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:23.344830 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:23.345350 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:23.346001 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.346021 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.347364 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.347368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.348750 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.348763 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.349535 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.349540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.349739 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.350345 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:23.350366 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:23.352749 (+ 2383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":115,"threads_started":1}
W20260806 18:40:23.354010 22824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:23.341924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:57077 (local address 127.8.241.254:34303)
0806 18:40:23.342425 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.342446 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.343025 (+ 579us) server_negotiation.cc:407] Connection header received
0806 18:40:23.344104 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.344115 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.344393 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.344665 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.346185 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.347213 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.348860 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.349365 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.349480 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.353086 (+ 3606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:57077: BlockingRecv error: recv got EOF from 127.8.241.193:57077 (error 108)
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":124,"threads_started":1}
W20260806 18:40:23.354126 22820 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:34303 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.241.254:34303: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:23.359460 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.354545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:33089)
0806 18:40:23.354708 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:23.354721 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:23.354786 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:23.355002 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:23.355004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:23.355126 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:23.355214 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.355218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.356195 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.356198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.356732 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.356740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.357303 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.357306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.357381 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.357814 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:23.357826 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:23.359365 (+ 1539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260806 18:40:23.359617 22824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:23.354654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:33089 (local address 127.8.241.254:34303)
0806 18:40:23.354765 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.354770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.354784 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:23.354863 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.354866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.354912 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.354992 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.355301 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.356090 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.356826 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.357194 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.357290 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.359511 (+ 2221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:33089: BlockingRecv error: recv got EOF from 127.8.241.193:33089 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:23.364841 22826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.360200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:44309)
0806 18:40:23.360350 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:23.360363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:23.360427 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:23.360621 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:23.360623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:23.360739 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:23.360809 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.360813 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.361769 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.361772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.362273 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.362280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.362801 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.362804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.362855 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.363257 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:23.363268 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:23.364751 (+ 1483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91}
W20260806 18:40:23.364987 22824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:23.360306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:44309 (local address 127.8.241.254:34303)
0806 18:40:23.360413 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.360417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.360430 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:23.360495 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.360498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.360538 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.360604 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.360896 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.361683 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.362374 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.362714 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.362807 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.364889 (+ 2082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:44309: BlockingRecv error: recv got EOF from 127.8.241.193:44309 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 18:40:23.365106 22820 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:40:23.458704 22827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:23.458963 22827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:23.459005 22827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:23.462891 22827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:23.462944 22827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:23.462998 22827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:23.468043 22827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.194.keytab
--principal=oryx/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:34413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=42073
--tserver_master_addrs=127.8.241.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:23.469137 22827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:23.470232 22827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:23.480291 22834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:23.480505 22838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:23.480274 22841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:23.480966 22827 server_base.cc:1034] running on GCE node
I20260806 18:40:23.481436 22827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:23.482164 22827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:23.485435 22827 hybrid_clock.cc:648] HybridClock initialized: now 1786041623485397 us; error 43 us; skew 500 ppm
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041623, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:23.488193 22827 init.cc:377] Logged in from keytab as oryx/127.8.241.194@KRBTEST.COM (short username oryx)
I20260806 18:40:23.489210 22827 webserver.cc:460] Webserver started at http://127.8.241.194:42073/ using document root <none> and password file <none>
I20260806 18:40:23.489771 22827 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:23.489848 22827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:23.493639 22827 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 18:40:23.495684 22850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.496917 22827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:23.497048 22827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "b2f72aa417be478d894b811b7a8cf896"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "2361eb887ee0d7aa68e8032958567f73"
server_key_iv: "81abf0139c86618a1921bce2b013508b"
server_key_version: "encryptionkey@0"
I20260806 18:40:23.497581 22827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:23.515879 22827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:23.518615 22827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:23.518800 22827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:23.519407 22827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:23.520237 22827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:23.520285 22827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.520380 22827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:23.520409 22827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.528775 22827 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:34413
I20260806 18:40:23.528798 23022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:34413 every 8 connection(s)
I20260806 18:40:23.529834 22827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:23.539713 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 22827
W20260806 18:40:23.540170 22824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:23.531619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:58343 (local address 127.8.241.254:34303)
0806 18:40:23.531749 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.531753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.532821 (+ 1068us) server_negotiation.cc:407] Connection header received
0806 18:40:23.533685 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.533687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.533737 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.533810 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.534918 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.535427 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.536586 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.536793 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.536866 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.540057 (+ 3191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:58343: BlockingRecv error: recv got EOF from 127.8.241.194:58343 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:40:23.540236 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:45381
--local_ip_for_outbound_sockets=127.8.241.195
--tserver_master_addrs=127.8.241.254:34303
--webserver_port=40143
--webserver_interface=127.8.241.195
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.241.195
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:40:23.540840 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.532012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:58343)
0806 18:40:23.532670 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:23.532710 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:23.533506 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:23.533966 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:23.533973 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:23.534329 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:23.534802 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.534816 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.535577 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.535581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.536477 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.536488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.536906 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.536908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.537102 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.537613 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:23.537629 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:23.539834 (+ 2205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":114,"threads_started":1}
W20260806 18:40:23.541350 23023 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:34303 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.241.254:34303: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:23.547122 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.541858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:35923)
0806 18:40:23.541990 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:23.542005 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:23.542074 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:23.542317 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:23.542319 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:23.542454 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:23.542545 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.542550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.543560 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.543562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.544087 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.544094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.544668 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.544671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.544731 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.545211 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:23.545237 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:23.547025 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 18:40:23.547356 22824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:23.541950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:35923 (local address 127.8.241.254:34303)
0806 18:40:23.542097 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.542102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.542119 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:40:23.542165 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.542168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.542216 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.542309 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.542655 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.543458 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.544202 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.544533 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.544682 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.547208 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:35923: BlockingRecv error: recv got EOF from 127.8.241.194:35923 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:23.552117 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.547725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:42193)
0806 18:40:23.547871 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:23.547883 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:23.547966 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:23.548177 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:23.548180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:23.548319 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:23.548387 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.548392 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.549083 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.549086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.549588 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.549595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.549997 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.550000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.550074 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.550475 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:23.550487 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:23.551998 (+ 1511us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260806 18:40:23.552224 22824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:23.547824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:42193 (local address 127.8.241.254:34303)
0806 18:40:23.547939 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.547944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.547956 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:40:23.548048 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.548051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.548092 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.548153 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.548494 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.548976 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.549693 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.549901 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.549984 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.552125 (+ 2141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:42193: BlockingRecv error: recv got EOF from 127.8.241.194:42193 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:40:23.552325 23023 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:40:23.658895 23028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:23.659175 23028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:23.659252 23028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:23.663245 23028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:23.663316 23028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:23.663403 23028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:23.668699 23028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/krb5kdc/oryx_127.8.241.195.keytab
--principal=oryx/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:45381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=40143
--tserver_master_addrs=127.8.241.254:34303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:23.669862 23028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:23.671053 23028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:23.680706 23034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:23.680707 23033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:23.680763 23036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:23.681049 23028 server_base.cc:1034] running on GCE node
I20260806 18:40:23.681473 23028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:23.682045 23028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:23.683202 23028 hybrid_clock.cc:648] HybridClock initialized: now 1786041623683181 us; error 32 us; skew 500 ppm
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041623, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:23.685762 23028 init.cc:377] Logged in from keytab as oryx/127.8.241.195@KRBTEST.COM (short username oryx)
I20260806 18:40:23.686590 23028 webserver.cc:460] Webserver started at http://127.8.241.195:40143/ using document root <none> and password file <none>
I20260806 18:40:23.687122 23028 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:23.687206 23028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:23.690999 23028 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:40:23.693025 23048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.694180 23028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:23.694288 23028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "0415179bd79a4fdbb82fac5aa313550a"
format_stamp: "Formatted at 2026-08-06 18:40:22 on dist-test-slave-k5rr"
server_key: "0851377d4b8a5b68199faea239c239aa"
server_key_iv: "d949f74035acb15b14b5ae79a603c23c"
server_key_version: "encryptionkey@0"
I20260806 18:40:23.694823 23028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:23.713173 23028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:23.715946 23028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:23.716130 23028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:23.716738 23028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:23.717716 23028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:23.717778 23028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.717856 23028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:23.717898 23028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:23.726563 23028 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:45381
I20260806 18:40:23.726567 23218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:45381 every 8 connection(s)
I20260806 18:40:23.727571 23028 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:23.737583 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 23028
W20260806 18:40:23.738072 22824 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:23.729370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:36161 (local address 127.8.241.254:34303)
0806 18:40:23.729496 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.729500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.730562 (+ 1062us) server_negotiation.cc:407] Connection header received
0806 18:40:23.731476 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.731479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.731539 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.731614 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.732802 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.733370 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.734487 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.734698 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.734787 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.737952 (+ 3165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:36161: BlockingRecv error: recv got EOF from 127.8.241.195:36161 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:40:23.738763 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.729759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:36161)
0806 18:40:23.730414 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:23.730459 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:23.731254 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:23.731813 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:23.731821 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:23.732178 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:23.732657 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.732670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.733500 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.733503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.734364 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.734374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.734839 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.734844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.735025 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.735544 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:23.735561 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:23.737805 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":122,"threads_started":1}
W20260806 18:40:23.739218 23220 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:34303 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.241.254:34303: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:23.744853 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.739608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:50869)
0806 18:40:23.739741 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:23.739751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:23.739807 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:23.740119 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:23.740122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:23.740261 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:23.740343 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.740348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.741077 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.741081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.741884 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.741894 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.742382 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.742386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.742476 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.742893 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:23.742906 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:23.744730 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83}
W20260806 18:40:23.744982 23224 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:23.739702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:50869 (local address 127.8.241.254:34303)
0806 18:40:23.739940 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.739944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.739955 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:40:23.739983 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.739985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.740017 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.740089 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.740458 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.740944 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.741995 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.742259 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.742352 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.744864 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:50869: BlockingRecv error: recv got EOF from 127.8.241.195:50869 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260806 18:40:23.747406 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.738680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51214 (local address 127.8.241.254:34303)
0806 18:40:23.738807 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.738811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.738824 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:23.738861 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.738864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.738905 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.738972 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.740183 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.740669 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.741811 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.742059 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.742148 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.743554 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:23.743562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:23.743586 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:23.743620 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:23.745376 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:23.745843 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:23.745848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:23.745853 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:23.745975 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:23.746236 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:23.746239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:23.746240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:23.746597 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:23.746798 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:23.747072 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 18:40:23.747304 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:23 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:23.750176 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:23.745525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:51805)
0806 18:40:23.745615 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:23.745628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:23.745702 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:23.745913 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:23.745916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:23.746060 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:23.746128 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.746133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.746811 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.746815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.747389 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:23.747399 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.747801 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.747805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:23.747876 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.748276 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:23.748291 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:23.750065 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 18:40:23.750250 23224 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:23.745596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:51805 (local address 127.8.241.254:34303)
0806 18:40:23.745741 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:23.745745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:23.745757 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:40:23.745789 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:23.745791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:23.745827 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:23.745885 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:23.746230 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.746714 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.747493 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:23.747707 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:23.747789 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:23.750159 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:51805: BlockingRecv error: recv got EOF from 127.8.241.195:51805 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 18:40:23.750461 23220 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 18:40:24 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:24 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:24.371469 23225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:24.365771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:40667)
0806 18:40:24.366039 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:24.366053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:24.366145 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:24.366377 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:24.366379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:24.366570 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:24.366692 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:24.366697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.367488 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.367491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:24.368239 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:24.368251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.368698 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.368701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:24.368780 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:24.369322 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:24.369337 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:24.371327 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
W20260806 18:40:24.371594 23226 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:24.365819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:40667 (local address 127.8.241.254:34303)
0806 18:40:24.366089 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:24.366093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:24.366106 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:24.366221 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:24.366223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:24.366278 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:24.366353 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:24.366815 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.367371 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.368351 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.368609 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.368686 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:24.371480 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:40667: BlockingRecv error: recv got EOF from 127.8.241.193:40667 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
Aug 06 18:40:24 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:24 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:24.558105 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:24.552947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:39251)
0806 18:40:24.553255 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:24.553270 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:24.553372 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:24.553654 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:24.553657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:24.553888 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:24.554007 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:24.554014 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.554725 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.554728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:24.555304 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:24.555313 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.555677 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.555679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:24.555740 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:24.556249 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:24.556264 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:24.557967 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":125,"threads_started":1}
W20260806 18:40:24.558207 23226 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:24.553053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:39251 (local address 127.8.241.254:34303)
0806 18:40:24.553208 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:24.553212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:24.553336 (+ 124us) server_negotiation.cc:407] Connection header received
0806 18:40:24.553445 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:24.553447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:24.553492 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:24.553569 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:24.554108 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.554612 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.555393 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.555597 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.555662 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:24.558104 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:39251: BlockingRecv error: recv got EOF from 127.8.241.194:39251 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 18:40:24 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:24 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:24.755961 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:24.751012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:50973)
0806 18:40:24.751280 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:24.751293 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:24.751390 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:24.751607 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:24.751609 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:24.751784 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:24.751879 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:24.751883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.752561 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.752563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:24.753155 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:24.753165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.753527 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.753530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:24.753585 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:24.754072 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:24.754083 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:24.755830 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":93,"threads_started":1}
W20260806 18:40:24.756090 23226 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:24.751088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:50973 (local address 127.8.241.254:34303)
0806 18:40:24.751235 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:24.751238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:24.751352 (+ 114us) server_negotiation.cc:407] Connection header received
0806 18:40:24.751480 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:24.751482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:24.751529 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:24.751593 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:24.751959 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.752479 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.753234 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:24.753448 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:24.753519 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:24.755979 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:50973: BlockingRecv error: recv got EOF from 127.8.241.195:50973 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 18:40:25 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:25 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:25.377803 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:25.372207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:36639)
0806 18:40:25.372510 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:25.372525 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:25.372627 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:25.372854 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:25.372856 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:25.373083 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:25.373186 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:25.373192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.374017 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.374019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:25.374643 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:25.374657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.375177 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.375182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:25.375266 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:25.375767 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:25.375779 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:25.377679 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":92,"threads_started":1}
W20260806 18:40:25.377960 23232 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:25.372286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:36639 (local address 127.8.241.254:34303)
0806 18:40:25.372575 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:25.372578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:25.372592 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:25.372705 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:25.372707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:25.372751 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:25.372827 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:25.373336 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.373923 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.374787 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.375050 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.375154 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:25.377814 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:36639: BlockingRecv error: recv got EOF from 127.8.241.193:36639 (error 108)
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":98,"threads_started":1}
Aug 06 18:40:25 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:25 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:25.565063 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:25.558953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:53509)
0806 18:40:25.559208 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:25.559222 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:25.559326 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:25.559582 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:25.559584 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:25.559754 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:25.559858 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:25.559863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.560926 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.560929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:25.561529 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:25.561538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.562125 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.562129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:25.562214 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:25.562771 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:25.562786 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:25.564881 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
W20260806 18:40:25.565331 23232 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:25.559045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:53509 (local address 127.8.241.254:34303)
0806 18:40:25.559200 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:25.559205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:25.559290 (+ 85us) server_negotiation.cc:407] Connection header received
0806 18:40:25.559409 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:25.559413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:25.559467 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:25.559570 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:25.559963 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.560824 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.561637 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.561988 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.562098 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:25.565099 (+ 3001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:53509: BlockingRecv error: recv got EOF from 127.8.241.194:53509 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 18:40:25 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:25 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:25.762295 23236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:25.756767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:33735)
0806 18:40:25.757095 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:25.757112 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:25.757227 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:25.757474 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:25.757477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:25.757683 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:25.757798 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:25.757805 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.758575 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.758579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:25.759306 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:25.759318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.759843 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.759846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:25.759919 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:25.760362 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:25.760373 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:25.762169 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":103,"threads_started":1}
W20260806 18:40:25.762398 23232 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:25.756960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:33735 (local address 127.8.241.254:34303)
0806 18:40:25.757106 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:25.757111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:25.757166 (+ 55us) server_negotiation.cc:407] Connection header received
0806 18:40:25.757315 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:25.757318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:25.757367 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:25.757430 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:25.757913 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.758463 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.759426 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:25.759712 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:25.759817 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:25.762295 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:33735: BlockingRecv error: recv got EOF from 127.8.241.195:33735 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 18:40:26 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:26 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:26.383976 23238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:26.378578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:48489)
0806 18:40:26.378927 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:26.378942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:26.379067 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:26.379334 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:26.379338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:26.379545 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:26.379647 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:26.379653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.380450 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.380454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:26.381049 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:26.381061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.381492 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.381496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:26.381583 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:26.382068 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:26.382080 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:26.383837 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":88,"threads_started":1}
W20260806 18:40:26.384238 23239 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:26.378664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:48489 (local address 127.8.241.254:34303)
0806 18:40:26.378915 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:26.378920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:26.379020 (+ 100us) server_negotiation.cc:407] Connection header received
0806 18:40:26.379160 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:26.379163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:26.379210 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:26.379287 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:26.379772 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.380333 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.381161 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.381383 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.381473 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:26.384006 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:48489: BlockingRecv error: recv got EOF from 127.8.241.193:48489 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":61,"threads_started":1}
Aug 06 18:40:26 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:26 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:26.575299 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:26.565791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:37631)
0806 18:40:26.566040 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:26.566055 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:26.566162 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:26.566375 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:26.566377 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:26.566569 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:26.566667 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:26.566672 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.567344 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.567346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:26.567889 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:26.567895 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.568335 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.568338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:26.568399 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:26.573338 (+ 4939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:26.573354 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:26.575144 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":70,"threads_started":1}
W20260806 18:40:26.575392 23239 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:26.565857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:37631 (local address 127.8.241.254:34303)
0806 18:40:26.565996 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:26.566000 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:26.566120 (+ 120us) server_negotiation.cc:407] Connection header received
0806 18:40:26.566246 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:26.566248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:26.566293 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:26.566348 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:26.566753 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.567269 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.567990 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.568193 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.568290 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:26.575290 (+ 7000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:37631: BlockingRecv error: recv got EOF from 127.8.241.194:37631 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 18:40:26 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:26 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:26.768894 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:26.763046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:46899)
0806 18:40:26.763367 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:26.763385 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:26.763521 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:26.763806 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:26.763810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:26.764021 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:26.764127 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:26.764134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.764932 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.764936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:26.765550 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:26.765561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.766030 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.766035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:26.766120 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:26.766616 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:26.766628 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:26.768726 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":88,"threads_started":1}
W20260806 18:40:26.768973 23239 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:26.763177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:46899 (local address 127.8.241.254:34303)
0806 18:40:26.763361 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:26.763367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:26.763467 (+ 100us) server_negotiation.cc:407] Connection header received
0806 18:40:26.763636 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:26.763638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:26.763689 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:26.763763 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:26.764260 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.764809 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.765682 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:26.765912 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:26.766010 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:26.768860 (+ 2850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:46899: BlockingRecv error: recv got EOF from 127.8.241.195:46899 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 06 18:40:27 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:27 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:27.390818 23242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:27.384745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:33643)
0806 18:40:27.385127 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:27.385140 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:27.385232 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:27.385486 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:27.385488 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:27.385658 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:27.385757 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:27.385762 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.386827 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.386829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:27.387395 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:27.387403 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.387987 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.387990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:27.388050 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:27.388734 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:27.388749 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:27.390692 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":83,"threads_started":1}
W20260806 18:40:27.391016 23243 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:27.384863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:33643 (local address 127.8.241.254:34303)
0806 18:40:27.385169 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:27.385174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:27.385191 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:40:27.385307 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:27.385310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:27.385365 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:27.385472 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:27.385857 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.386726 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.387507 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.387877 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.387977 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:27.390861 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:33643: BlockingRecv error: recv got EOF from 127.8.241.193:33643 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":84,"threads_started":1}
Aug 06 18:40:27 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:27 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:27.581540 23245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:27.576120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:48735)
0806 18:40:27.576404 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:27.576418 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:27.576524 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:27.576788 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:27.576791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:27.576990 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:27.577161 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:27.577169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.578018 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.578021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:27.578668 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:27.578678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.579110 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.579113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:27.579177 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:27.579634 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:27.579645 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:27.581407 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":93,"threads_started":1}
W20260806 18:40:27.581669 23243 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:27.576199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:48735 (local address 127.8.241.254:34303)
0806 18:40:27.576326 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:27.576330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:27.576483 (+ 153us) server_negotiation.cc:407] Connection header received
0806 18:40:27.576597 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:27.576600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:27.576643 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:27.576710 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:27.577281 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.577904 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.578773 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.578999 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.579079 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:27.581549 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:48735: BlockingRecv error: recv got EOF from 127.8.241.194:48735 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 18:40:27 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:27 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:27.775015 23246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:27.769773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:41161)
0806 18:40:27.770068 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:27.770082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:27.770186 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:27.770425 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:27.770428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:27.770620 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:27.770722 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:27.770728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.771419 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.771423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:27.772147 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:27.772160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.772662 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.772665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:27.772731 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:27.773187 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:27.773199 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:27.774867 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":84,"threads_started":1}
W20260806 18:40:27.775096 23243 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:27.769831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:41161 (local address 127.8.241.254:34303)
0806 18:40:27.769975 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:27.769980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:27.770145 (+ 165us) server_negotiation.cc:407] Connection header received
0806 18:40:27.770259 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:27.770261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:27.770308 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:27.770371 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:27.770809 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.771313 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.772260 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:27.772543 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:27.772621 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:27.774994 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:41161: BlockingRecv error: recv got EOF from 127.8.241.195:41161 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 18:40:28 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:28 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:28.396979 23247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:28.391714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:38107)
0806 18:40:28.391992 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:28.392006 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:28.392098 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:28.392324 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:28.392326 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:28.392527 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:28.392626 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:28.392632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.393331 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.393333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:28.393919 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:28.393928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.394346 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.394350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:28.394437 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:28.394998 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:28.395011 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:28.396809 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":83,"threads_started":1}
W20260806 18:40:28.397110 23248 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:28.391812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:38107 (local address 127.8.241.254:34303)
0806 18:40:28.392053 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:28.392057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:28.392072 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:40:28.392176 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:28.392178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:28.392221 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:28.392299 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:28.392714 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.393257 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.394005 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.394216 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.394300 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:28.396957 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:38107: BlockingRecv error: recv got EOF from 127.8.241.193:38107 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
Aug 06 18:40:28 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:28 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:28.587826 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:28.582357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:60231)
0806 18:40:28.582654 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:28.582667 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:28.582765 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:28.583027 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:28.583029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:28.583268 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:28.583375 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:28.583383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.584294 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.584297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:28.584901 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:28.584912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.585362 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.585365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:28.585423 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:28.585883 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:28.585893 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:28.587659 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":107,"threads_started":1}
W20260806 18:40:28.587910 23248 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:28.582508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:60231 (local address 127.8.241.254:34303)
0806 18:40:28.582648 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:28.582652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:28.582728 (+ 76us) server_negotiation.cc:407] Connection header received
0806 18:40:28.582855 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:28.582858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:28.582910 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:28.582998 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:28.583485 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.584172 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.585027 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.585247 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.585327 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:28.587808 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:60231: BlockingRecv error: recv got EOF from 127.8.241.194:60231 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 18:40:28 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:28 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:28.781066 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:28.775725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:58295)
0806 18:40:28.776040 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:28.776053 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:28.776152 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:28.776364 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:28.776367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:28.776550 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:28.776643 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:28.776648 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.777479 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.777483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:28.778046 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:28.778055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.778495 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.778499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:28.778584 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:28.779136 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:28.779152 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:28.780921 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":108,"threads_started":1}
W20260806 18:40:28.781195 23248 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:28.775830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:58295 (local address 127.8.241.254:34303)
0806 18:40:28.775969 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:28.775973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:28.776112 (+ 139us) server_negotiation.cc:407] Connection header received
0806 18:40:28.776224 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:28.776226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:28.776273 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:28.776337 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:28.776727 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.777377 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.778135 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:28.778361 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:28.778455 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:28.781086 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:58295: BlockingRecv error: recv got EOF from 127.8.241.195:58295 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 18:40:29 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:29 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:29.403245 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:29.397794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:44191)
0806 18:40:29.398053 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:29.398067 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:29.398158 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:29.398423 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:29.398426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:29.398647 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:29.398752 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:29.398758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.399629 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.399632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:29.400238 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:29.400249 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.400680 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.400683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:29.400745 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:29.401325 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:29.401337 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:29.403119 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
W20260806 18:40:29.403376 23255 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:29.397907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:44191 (local address 127.8.241.254:34303)
0806 18:40:29.398135 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:29.398138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:29.398152 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:29.398243 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:29.398245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:29.398288 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:29.398359 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:29.398866 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.399520 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.400331 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.400585 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.400682 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:29.403256 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:44191: BlockingRecv error: recv got EOF from 127.8.241.193:44191 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
Aug 06 18:40:29 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:29 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:29.593961 23256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:29.588681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:43525)
0806 18:40:29.588962 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:29.588975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:29.589095 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:29.589321 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:29.589324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:29.589509 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:29.589607 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:29.589613 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.590274 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.590276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:29.590817 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:29.590823 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.591176 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.591178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:29.591233 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:29.591738 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:29.591750 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:29.593801 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":91,"threads_started":1}
W20260806 18:40:29.594058 23255 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:29.588756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:43525 (local address 127.8.241.254:34303)
0806 18:40:29.588898 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:29.588902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:29.589054 (+ 152us) server_negotiation.cc:407] Connection header received
0806 18:40:29.589185 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:29.589187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:29.589232 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:29.589294 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:29.589693 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.590195 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.590894 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.591099 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.591164 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:29.593943 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:43525: BlockingRecv error: recv got EOF from 127.8.241.194:43525 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 18:40:29 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:29 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:29.786903 23257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:29.781802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:44823)
0806 18:40:29.782093 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:29.782105 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:29.782202 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:29.782423 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:29.782425 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:29.782624 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:29.782720 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:29.782726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.783404 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.783406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:29.783970 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:29.783978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.784356 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.784358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:29.784416 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:29.784907 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:29.784919 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:29.786748 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
W20260806 18:40:29.787019 23255 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:29.781890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:44823 (local address 127.8.241.254:34303)
0806 18:40:29.782021 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:29.782025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:29.782164 (+ 139us) server_negotiation.cc:407] Connection header received
0806 18:40:29.782274 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:29.782276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:29.782323 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:29.782398 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:29.782806 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.783326 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.784063 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:29.784274 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:29.784347 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:29.786901 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:44823: BlockingRecv error: recv got EOF from 127.8.241.195:44823 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 18:40:30 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:30 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:30.408978 23258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:30.404001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:38675)
0806 18:40:30.404358 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:30.404371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:30.404475 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:30.404698 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:30.404700 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:30.404884 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:30.405028 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:30.405036 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.405709 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.405711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:30.406269 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:30.406276 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.406626 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.406629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:30.406682 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:30.407143 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:30.407155 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:30.408824 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":90,"threads_started":1}
W20260806 18:40:30.409157 23260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:30.404079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:38675 (local address 127.8.241.254:34303)
0806 18:40:30.404314 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:30.404318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:30.404435 (+ 117us) server_negotiation.cc:407] Connection header received
0806 18:40:30.404550 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:30.404552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:30.404595 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:30.404671 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:30.405123 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.405637 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.406348 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.406552 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.406616 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:30.408998 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:38675: BlockingRecv error: recv got EOF from 127.8.241.193:38675 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":66,"threads_started":1}
Aug 06 18:40:30 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:30 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:30.599982 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:30.594708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:53263)
0806 18:40:30.595018 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:30.595032 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:30.595134 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:30.595342 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:30.595344 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:30.595535 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:30.595630 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:30.595636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.596327 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.596329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:30.596885 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:30.596891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.597315 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.597318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:30.597377 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:30.597889 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:30.597902 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:30.599846 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":74,"threads_started":1}
W20260806 18:40:30.600083 23260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:30.594802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:53263 (local address 127.8.241.254:34303)
0806 18:40:30.594988 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:30.594993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:30.595094 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:40:30.595206 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:30.595208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:30.595254 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:30.595334 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:30.595718 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.596247 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.596970 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.597228 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.597302 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:30.599980 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:53263: BlockingRecv error: recv got EOF from 127.8.241.194:53263 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 18:40:30 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:30 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:30.794075 23262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:30.787733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:46563)
0806 18:40:30.788258 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:30.788274 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:30.788387 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:30.788797 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:30.788801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:30.789038 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:30.789161 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:30.789169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.790212 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.790215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:30.790763 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:30.790771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.791334 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.791336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:30.791393 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:30.791949 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:30.791962 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:30.793954 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":97,"threads_started":1}
W20260806 18:40:30.794353 23260 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:30.787841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:46563 (local address 127.8.241.254:34303)
0806 18:40:30.788001 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:30.788007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:30.788335 (+ 328us) server_negotiation.cc:407] Connection header received
0806 18:40:30.788612 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:30.788614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:30.788665 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:30.788740 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:30.789259 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.790100 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.790872 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:30.791231 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:30.791341 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:30.794131 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:46563: BlockingRecv error: recv got EOF from 127.8.241.195:46563 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 18:40:31 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:31 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:31.415377 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:31.409777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:54911)
0806 18:40:31.410069 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:31.410084 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:31.410181 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:31.410621 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:31.410624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:31.410812 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:31.410940 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:31.410948 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.411910 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.411913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:31.412490 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:31.412500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.412885 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.412888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:31.412945 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:31.413487 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:31.413499 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:31.415248 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
W20260806 18:40:31.415518 23267 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:31.409940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:54911 (local address 127.8.241.254:34303)
0806 18:40:31.410187 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:31.410190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:31.410204 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:31.410298 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:31.410301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:31.410357 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:31.410446 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:31.411071 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.411806 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.412589 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.412806 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.412876 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:31.415376 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:54911: BlockingRecv error: recv got EOF from 127.8.241.193:54911 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
Aug 06 18:40:31 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:31 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:31.612986 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:31.600853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:33843)
0806 18:40:31.601197 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:31.601209 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:31.601321 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:31.601558 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:31.601561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:31.601781 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:31.601881 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:31.601887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.602775 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.602779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:31.603396 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:31.603406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.603865 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.603869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:31.603962 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:31.611046 (+ 7084us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:31.611061 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:31.612834 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":75,"threads_started":1}
W20260806 18:40:31.613135 23267 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:31.600873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:33843 (local address 127.8.241.254:34303)
0806 18:40:31.601022 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:31.601028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:31.601275 (+ 247us) server_negotiation.cc:407] Connection header received
0806 18:40:31.601405 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:31.601408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:31.601467 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:31.601539 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:31.601994 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.602670 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.603512 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.603743 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.603830 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:31.612984 (+ 9154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:33843: BlockingRecv error: recv got EOF from 127.8.241.194:33843 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 18:40:31 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:31 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:31.799909 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:31.794782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:50395)
0806 18:40:31.795053 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:31.795066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:31.795165 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:31.795374 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:31.795376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:31.795566 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:31.795665 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:31.795671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.796336 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.796338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:31.796897 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:31.796904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.797319 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.797322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:31.797385 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:31.797866 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:31.797877 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:31.799778 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":87,"threads_started":1}
W20260806 18:40:31.800186 23267 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:31.794853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:50395 (local address 127.8.241.254:34303)
0806 18:40:31.794980 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:31.794984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:31.795125 (+ 141us) server_negotiation.cc:407] Connection header received
0806 18:40:31.795242 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:31.795244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:31.795290 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:31.795347 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:31.795751 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.796257 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.796979 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:31.797210 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:31.797280 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:31.799928 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:50395: BlockingRecv error: recv got EOF from 127.8.241.195:50395 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 18:40:32 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:32 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:32.421132 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:32.416132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:41589)
0806 18:40:32.416405 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:32.416418 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:32.416518 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:32.416750 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:32.416753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:32.416937 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:32.417078 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:32.417085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.417773 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.417775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:32.418316 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:32.418322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.418671 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.418674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:32.418730 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:32.419183 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:32.419194 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:32.420954 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":89,"threads_started":1}
W20260806 18:40:32.421463 23276 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:32.416204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:41589 (local address 127.8.241.254:34303)
0806 18:40:32.416457 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:32.416460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:32.416473 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:40:32.416595 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:32.416597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:32.416641 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:32.416717 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:32.417167 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.417692 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.418391 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.418596 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.418661 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:32.421124 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:41589: BlockingRecv error: recv got EOF from 127.8.241.193:41589 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":60,"threads_started":1}
Aug 06 18:40:32 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:32 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:32.619513 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:32.613882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:37845)
0806 18:40:32.614171 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:32.614187 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:32.614309 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:32.614574 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:32.614577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:32.614783 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:32.614880 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:32.614887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.615635 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.615639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:32.616193 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:32.616201 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.616660 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.616664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:32.616752 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:32.617328 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:32.617342 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:32.619335 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":99,"threads_started":1}
W20260806 18:40:32.619592 23276 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:32.614019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:37845 (local address 127.8.241.254:34303)
0806 18:40:32.614171 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:32.614176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:32.614251 (+ 75us) server_negotiation.cc:407] Connection header received
0806 18:40:32.614411 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:32.614413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:32.614462 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:32.614530 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:32.615010 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.615529 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.616310 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.616531 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.616619 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:32.619483 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:37845: BlockingRecv error: recv got EOF from 127.8.241.194:37845 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 18:40:32 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:32 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:32.806277 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:32.800710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:35043)
0806 18:40:32.801034 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:32.801048 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:32.801159 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:32.801424 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:32.801427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:32.801625 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:32.801726 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:32.801733 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.802624 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.802628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:32.803270 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:32.803282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.803743 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.803747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:32.803828 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:32.804298 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:32.804310 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:32.806135 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":101,"threads_started":1}
W20260806 18:40:32.806391 23276 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:32.800793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:35043 (local address 127.8.241.254:34303)
0806 18:40:32.800967 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:32.800974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:32.801114 (+ 140us) server_negotiation.cc:407] Connection header received
0806 18:40:32.801264 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:32.801267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:32.801315 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:32.801382 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:32.801845 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.802512 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.803388 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:32.803616 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:32.803704 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:32.806269 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:35043: BlockingRecv error: recv got EOF from 127.8.241.195:35043 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 18:40:33 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:33 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:33.426798 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:33.421902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:59951)
0806 18:40:33.422183 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:33.422197 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:33.422287 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:33.422521 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:33.422523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:33.422709 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:33.422808 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:33.422813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.423494 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.423496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:33.424061 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:33.424068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.424426 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.424429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:33.424486 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:33.424958 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:33.424969 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:33.426657 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":79,"threads_started":1}
W20260806 18:40:33.426919 23284 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:33.421989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:59951 (local address 127.8.241.254:34303)
0806 18:40:33.422248 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:33.422252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:33.422266 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:40:33.422366 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:33.422368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:33.422410 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:33.422497 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:33.422894 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.423410 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.424140 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.424347 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.424413 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:33.426801 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:59951: BlockingRecv error: recv got EOF from 127.8.241.193:59951 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
Aug 06 18:40:33 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:33 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:33.625537 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:33.620423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:50377)
0806 18:40:33.620721 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:33.620734 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:33.620834 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:33.621092 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:33.621094 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:33.621274 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:33.621371 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:33.621376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.622065 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.622067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:33.622608 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:33.622614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.622979 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.622981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:33.623036 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:33.623575 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:33.623587 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:33.625417 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":87,"threads_started":1}
W20260806 18:40:33.625670 23284 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:33.620510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:50377 (local address 127.8.241.254:34303)
0806 18:40:33.620698 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:33.620701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:33.620799 (+ 98us) server_negotiation.cc:407] Connection header received
0806 18:40:33.620908 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:33.620910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:33.620970 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:33.621062 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:33.621464 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.621986 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.622687 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.622887 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.622964 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:33.625560 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:50377: BlockingRecv error: recv got EOF from 127.8.241.194:50377 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 18:40:33 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:33 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:33.812374 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:33.807187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:59639)
0806 18:40:33.807470 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:33.807484 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:33.807582 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:33.807784 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:33.807786 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:33.807975 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:33.808074 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:33.808080 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.808809 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.808813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:33.809432 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:33.809443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.809854 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.809857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:33.809925 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:33.810441 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:33.810453 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:33.812248 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
W20260806 18:40:33.812507 23284 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:33.807307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:59639 (local address 127.8.241.254:34303)
0806 18:40:33.807434 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:33.807437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:33.807544 (+ 107us) server_negotiation.cc:407] Connection header received
0806 18:40:33.807646 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:33.807648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:33.807694 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:33.807756 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:33.808162 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.808668 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.809533 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:33.809751 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:33.809825 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:33.812383 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:59639: BlockingRecv error: recv got EOF from 127.8.241.195:59639 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 18:40:34 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:34 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:34.433176 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:34.427704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:37197)
0806 18:40:34.427996 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:34.428011 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:34.428112 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:34.428343 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:34.428345 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:34.428542 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:34.428646 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:34.428651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.429512 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.429516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:34.430274 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:34.430288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.430739 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.430743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:34.430824 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:34.431299 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:34.431310 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:34.433048 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
W20260806 18:40:34.433300 23291 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:34.427786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:37197 (local address 127.8.241.254:34303)
0806 18:40:34.428041 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:34.428044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:34.428062 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:40:34.428195 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:34.428197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:34.428242 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:34.428317 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:34.428735 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.429384 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.430376 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.430625 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.430715 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:34.433183 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:37197: BlockingRecv error: recv got EOF from 127.8.241.193:37197 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":61,"threads_started":1}
Aug 06 18:40:34 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:34 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:34.631150 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:34.626243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:33213)
0806 18:40:34.626507 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:34.626521 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:34.626625 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:34.626832 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:34.626835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:34.627028 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:34.627126 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:34.627132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.627817 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.627819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:34.628381 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:34.628389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.628765 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.628768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:34.628822 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:34.629303 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:34.629314 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:34.631028 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
W20260806 18:40:34.631294 23291 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:34.626296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:33213 (local address 127.8.241.254:34303)
0806 18:40:34.626440 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:34.626453 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:34.626584 (+ 131us) server_negotiation.cc:407] Connection header received
0806 18:40:34.626703 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:34.626705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:34.626751 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:34.626807 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:34.627210 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.627732 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.628472 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.628688 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.628755 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:34.631178 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:33213: BlockingRecv error: recv got EOF from 127.8.241.194:33213 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 18:40:34 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:34 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:34.818266 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:34.813122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:48127)
0806 18:40:34.813414 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:34.813426 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:34.813543 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:34.813788 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:34.813791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:34.814009 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:34.814106 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:34.814113 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.814852 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.814856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:34.815415 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:34.815424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.815855 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.815859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:34.815950 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:34.816425 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:34.816435 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:34.818118 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":104,"threads_started":1}
W20260806 18:40:34.818368 23291 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:34.813154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:48127 (local address 127.8.241.254:34303)
0806 18:40:34.813269 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:34.813274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:34.813497 (+ 223us) server_negotiation.cc:407] Connection header received
0806 18:40:34.813635 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:34.813638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:34.813685 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:34.813751 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:34.814220 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.814747 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.815533 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:34.815752 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:34.815837 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:34.818265 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:48127: BlockingRecv error: recv got EOF from 127.8.241.195:48127 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 18:40:35 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:35 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:35.439317 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:35.434047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:46297)
0806 18:40:35.434419 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:35.434434 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:35.434561 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:35.434848 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:35.434851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:35.435109 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:35.435229 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:35.435235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.435970 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.435972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:35.436515 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:35.436522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.436903 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.436905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:35.436975 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:35.437477 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:35.437488 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:35.439192 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":85,"threads_started":1}
W20260806 18:40:35.439482 23301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:35.434147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:46297 (local address 127.8.241.254:34303)
0806 18:40:35.434411 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:35.434414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:35.434496 (+ 82us) server_negotiation.cc:407] Connection header received
0806 18:40:35.434663 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:35.434666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:35.434723 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:35.434801 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:35.435339 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.435877 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.436599 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.436819 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.436894 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:35.439332 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:46297: BlockingRecv error: recv got EOF from 127.8.241.193:46297 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":74,"threads_started":1}
Aug 06 18:40:35 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:35 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:35.637288 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:35.631901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:33015)
0806 18:40:35.632178 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:35.632194 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:35.632308 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:35.632560 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:35.632563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:35.632764 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:35.632862 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:35.632869 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.633664 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.633668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:35.634222 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:35.634231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.634705 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.634709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:35.634794 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:35.635325 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:35.635337 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:35.637172 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
W20260806 18:40:35.637415 23301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:35.632037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:33015 (local address 127.8.241.254:34303)
0806 18:40:35.632180 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:35.632185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:35.632258 (+ 73us) server_negotiation.cc:407] Connection header received
0806 18:40:35.632392 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:35.632395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:35.632451 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:35.632516 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:35.632979 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.633555 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.634331 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.634596 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.634685 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:35.637307 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:33015: BlockingRecv error: recv got EOF from 127.8.241.194:33015 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 06 18:40:35 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:35 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:35.824113 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:35.819044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:46647)
0806 18:40:35.819293 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:35.819306 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:35.819405 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:35.819630 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:35.819632 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:35.819815 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:35.819918 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:35.819923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.820619 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.820621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:35.821197 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:35.821205 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.821697 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.821700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:35.821769 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:35.822263 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:35.822275 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:35.823981 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
W20260806 18:40:35.824257 23301 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:35.819097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:46647 (local address 127.8.241.254:34303)
0806 18:40:35.819242 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:35.819245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:35.819367 (+ 122us) server_negotiation.cc:407] Connection header received
0806 18:40:35.819486 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:35.819488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:35.819536 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:35.819602 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:35.820015 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.820538 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.821331 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:35.821564 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:35.821650 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:35.824127 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:46647: BlockingRecv error: recv got EOF from 127.8.241.195:46647 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 18:40:36 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:36 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:36.445679 23304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:36.440153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:45399)
0806 18:40:36.440420 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:36.440433 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:36.440526 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:36.440747 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:36.440749 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:36.440930 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:36.441074 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:36.441080 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.441787 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.441789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:36.442335 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:36.442342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.442790 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.442792 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:36.442848 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:36.443364 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:36.443377 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:36.445523 (+ 2146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":80,"threads_started":1}
W20260806 18:40:36.445775 23305 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:36.440242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:45399 (local address 127.8.241.254:34303)
0806 18:40:36.440497 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:36.440500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:36.440516 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:40:36.440597 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:36.440599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:36.440647 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:36.440733 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:36.441167 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.441702 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.442462 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.442696 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.442784 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:36.445657 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:45399: BlockingRecv error: recv got EOF from 127.8.241.193:45399 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
Aug 06 18:40:36 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:36 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:36.647274 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:36.638087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:56483)
0806 18:40:36.638353 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:36.638368 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:36.638509 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:36.638804 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:36.638807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:36.639031 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:36.639176 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:36.639186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.639981 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.639984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:36.640541 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:36.640549 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.640931 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.640933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:36.640992 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:36.645046 (+ 4054us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:36.645063 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:36.647107 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
W20260806 18:40:36.647370 23305 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:36.638194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:56483 (local address 127.8.241.254:34303)
0806 18:40:36.638325 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:36.638329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:36.638438 (+ 109us) server_negotiation.cc:407] Connection header received
0806 18:40:36.638618 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:36.638621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:36.638679 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:36.638763 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:36.639294 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.639879 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.640627 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.640840 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.640906 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:36.647265 (+ 6359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:56483: BlockingRecv error: recv got EOF from 127.8.241.194:56483 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 18:40:36 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:36 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:36.829929 23307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:36.824788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:53027)
0806 18:40:36.825096 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:36.825110 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:36.825222 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:36.825484 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:36.825487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:36.825687 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:36.825784 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:36.825790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.826533 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.826537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:36.827097 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:36.827107 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.827536 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.827540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:36.827622 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:36.828091 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:36.828102 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:36.829811 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":85,"threads_started":1}
W20260806 18:40:36.830084 23305 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:36.824852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:53027 (local address 127.8.241.254:34303)
0806 18:40:36.825036 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:36.825040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:36.825178 (+ 138us) server_negotiation.cc:407] Connection header received
0806 18:40:36.825310 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:36.825312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:36.825361 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:36.825428 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:36.825905 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.826426 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.827214 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:36.827431 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:36.827517 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:36.829969 (+ 2452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:53027: BlockingRecv error: recv got EOF from 127.8.241.195:53027 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 18:40:37 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:37 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:37.451525 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:37.446599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:36999)
0806 18:40:37.446888 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:37.446901 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:37.446994 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:37.447216 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:37.447218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:37.447401 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:37.447519 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:37.447525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.448202 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.448205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:37.448762 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:37.448768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.449150 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.449153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:37.449211 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:37.449653 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:37.449663 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:37.451382 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
W20260806 18:40:37.451678 23309 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:37.446709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:36999 (local address 127.8.241.254:34303)
0806 18:40:37.447003 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:37.447006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:37.447042 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:40:37.447078 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:37.447079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:37.447123 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:37.447189 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:37.447609 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.448124 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.448841 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.449074 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.449137 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:37.451541 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:36999: BlockingRecv error: recv got EOF from 127.8.241.193:36999 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":69,"threads_started":1}
Aug 06 18:40:37 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:37 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:37.654035 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:37.648156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:58489)
0806 18:40:37.648463 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:37.648476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:37.648599 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:37.648866 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:37.648869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:37.649150 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:37.649277 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:37.649288 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.650315 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.650319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:37.651113 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:37.651125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.651612 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.651617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:37.651698 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:37.652141 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:37.652152 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:37.653847 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":122,"threads_started":1}
W20260806 18:40:37.654102 23309 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:37.648206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:58489 (local address 127.8.241.254:34303)
0806 18:40:37.648351 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:37.648356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:37.648548 (+ 192us) server_negotiation.cc:407] Connection header received
0806 18:40:37.648699 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:37.648703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:37.648758 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:37.648842 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:37.649399 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.650199 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.651222 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.651486 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.651575 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:37.653998 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:58489: BlockingRecv error: recv got EOF from 127.8.241.194:58489 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 18:40:37 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:37 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.195@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:37.836169 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:37.830725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.195:48213)
0806 18:40:37.831013 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:37.831025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:37.831129 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:37.831339 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:37.831341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:37.831530 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:37.831630 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:37.831636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.832363 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.832367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:37.832984 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:37.832996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.833463 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.833467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:37.833540 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:37.834066 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:37.834078 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:37.836003 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
W20260806 18:40:37.836251 23309 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:37.830834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:48213 (local address 127.8.241.254:34303)
0806 18:40:37.830965 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:37.830968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:37.831087 (+ 119us) server_negotiation.cc:407] Connection header received
0806 18:40:37.831196 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:37.831198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:37.831243 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:37.831311 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:37.831717 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.832228 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.833129 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:37.833337 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:37.833404 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:37.836144 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.195:48213: BlockingRecv error: recv got EOF from 127.8.241.195:48213 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 18:40:38 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:38 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.193@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:38.457638 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:38.452376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.193:58593)
0806 18:40:38.452680 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:38.452694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:38.452802 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:38.453065 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:38.453068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:38.453258 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:38.453364 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:38.453370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:38.454094 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:38.454097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:38.454661 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:38.454669 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:38.455072 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:38.455075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:38.455142 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:38.455600 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:38.455612 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:38.457461 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":108,"threads_started":1}
W20260806 18:40:38.457741 23318 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:38.452490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:58593 (local address 127.8.241.254:34303)
0806 18:40:38.452713 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:38.452716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:38.452769 (+ 53us) server_negotiation.cc:407] Connection header received
0806 18:40:38.452878 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:38.452880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:38.452922 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:38.453039 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:38.453454 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:38.453986 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:38.454751 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:38.454972 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:38.455037 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:38.457622 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.193:58593: BlockingRecv error: recv got EOF from 127.8.241.193:58593 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":60,"threads_started":1}
Aug 06 18:40:38 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:40:38 dist-test-slave-k5rr krb5kdc[21746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.241.194@KRBTEST.COM for oryx/127.8.241.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:40:38.660181 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:38.655008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34303 (local address 127.8.241.194:50451)
0806 18:40:38.655290 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:38.655304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:38.655414 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:38.655653 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:38.655656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:38.655860 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:38.655971 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:38.655977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:38.656719 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:38.656723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:38.657307 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:38.657316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:38.657758 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:38.657762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:38.657843 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:38.658284 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:38.658295 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:38.660023 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.241.254:34303: Server oryx/127.8.241.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":101,"threads_started":1}
W20260806 18:40:38.660290 23318 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:38.655091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:50451 (local address 127.8.241.254:34303)
0806 18:40:38.655251 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:38.655256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:38.655373 (+ 117us) server_negotiation.cc:407] Connection header received
0806 18:40:38.655504 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:38.655507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:38.655557 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:38.655621 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:38.656087 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:38.656618 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:38.657421 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:38.657656 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:38.657743 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:38.660174 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.241.194:50451: BlockingRecv error: recv got EOF from 127.8.241.194:50451 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:40:38.744750 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 22592
I20260806 18:40:38.760761 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 22827
I20260806 18:40:38.774101 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 23028
I20260806 18:40:38.785488 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 22487
2026-08-06T18:40:38Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18202 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T18:40:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:38Z Disabled control of system clock
I20260806 18:40:38.819293 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45931
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:45983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:45931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:40:38.932798 23328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:38.933102 23328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:38.933141 23328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:38.937247 23328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:38.937317 23328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:38.937331 23328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:38.937346 23328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:38.937361 23328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:38.942175 23328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:45983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:45931
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:38.943367 23328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:38.944844 23328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:40:38.953456 23328 server_base.cc:1034] running on GCE node
W20260806 18:40:38.953445 23338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:38.953442 23337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:38.953487 23342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:38.954247 23328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:38.955349 23328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:38.956552 23328 hybrid_clock.cc:648] HybridClock initialized: now 1786041638956525 us; error 35 us; skew 500 ppm
I20260806 18:40:38.958374 23328 webserver.cc:460] Webserver started at http://127.8.241.254:42263/ using document root <none> and password file <none>
I20260806 18:40:38.958959 23328 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:38.959029 23328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:38.959250 23328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:38.961349 23328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "cc69e5f2308343758d599c61f472b255"
format_stamp: "Formatted at 2026-08-06 18:40:38 on dist-test-slave-k5rr"
server_key: "3e48ba8a9493f7aca0176b782681b62e"
server_key_iv: "e3082c0bc83d9e62727525e3be233195"
server_key_version: "encryptionkey@0"
I20260806 18:40:38.961900 23328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "cc69e5f2308343758d599c61f472b255"
format_stamp: "Formatted at 2026-08-06 18:40:38 on dist-test-slave-k5rr"
server_key: "3e48ba8a9493f7aca0176b782681b62e"
server_key_iv: "e3082c0bc83d9e62727525e3be233195"
server_key_version: "encryptionkey@0"
I20260806 18:40:38.965555 23328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:40:38.967917 23350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:38.969127 23328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:38.969242 23328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "cc69e5f2308343758d599c61f472b255"
format_stamp: "Formatted at 2026-08-06 18:40:38 on dist-test-slave-k5rr"
server_key: "3e48ba8a9493f7aca0176b782681b62e"
server_key_iv: "e3082c0bc83d9e62727525e3be233195"
server_key_version: "encryptionkey@0"
I20260806 18:40:38.969342 23328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:38.983107 23328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:38.983767 23328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:38.983943 23328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:38.990796 23328 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:45931
I20260806 18:40:38.990810 23424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:45931 every 8 connection(s)
I20260806 18:40:38.991763 23328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:38.994704 23425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:38.995175 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 23328
I20260806 18:40:38.995277 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:38.995550 9159 external_mini_cluster.cc:1428] Setting key 146290a0beb9dd868a3d41520cab9c04
I20260806 18:40:39.001590 23425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255: Bootstrap starting.
I20260806 18:40:39.004215 23425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:39.005182 23425 log.cc:824] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:39.005869 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:38.996936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.8.241.254:45931)
0806 18:40:38.997667 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:38.997686 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:38.997730 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:40:38.998573 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:38.998598 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:38.999100 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:38.999484 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:38.999923 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.001391 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.002309 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.002986 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.003147 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:39.003854 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:39.003860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:39.003875 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:39.004370 (+ 495us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:39.004457 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:39.004489 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:39.004587 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:39.004702 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:40:39.004969 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":117,"threads_started":1}
I20260806 18:40:39.007236 23425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255: No bootstrap required, opened a new log
I20260806 18:40:39.010812 23425 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69e5f2308343758d599c61f472b255" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45931 } }
I20260806 18:40:39.011053 23425 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:39.011096 23425 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc69e5f2308343758d599c61f472b255, State: Initialized, Role: FOLLOWER
I20260806 18:40:39.011467 23425 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69e5f2308343758d599c61f472b255" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45931 } }
I20260806 18:40:39.011586 23425 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:39.011629 23425 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:39.011698 23425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:39.012529 23425 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69e5f2308343758d599c61f472b255" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45931 } }
I20260806 18:40:39.012825 23425 leader_election.cc:302] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc69e5f2308343758d599c61f472b255; no voters:
I20260806 18:40:39.013103 23425 leader_election.cc:288] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:39.013226 23432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:39.013423 23432 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [term 1 LEADER]: Becoming Leader. State: Replica: cc69e5f2308343758d599c61f472b255, State: Running, Role: LEADER
I20260806 18:40:39.013787 23432 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69e5f2308343758d599c61f472b255" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45931 } }
I20260806 18:40:39.014348 23425 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:39.015620 23433 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc69e5f2308343758d599c61f472b255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69e5f2308343758d599c61f472b255" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45931 } } }
I20260806 18:40:39.015637 23435 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc69e5f2308343758d599c61f472b255. Latest consensus state: current_term: 1 leader_uuid: "cc69e5f2308343758d599c61f472b255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc69e5f2308343758d599c61f472b255" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45931 } } }
I20260806 18:40:39.015868 23435 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:39.015851 23433 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:39.016649 23450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:39.019757 23450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:39.025324 23450 catalog_manager.cc:1347] Generated new cluster ID: 6cf4c2c0fba940268a140308d84343cb
I20260806 18:40:39.025379 23450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:39.039605 23450 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:39.040500 23450 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:39.046780 23450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc69e5f2308343758d599c61f472b255: Generated new TSK 0
I20260806 18:40:39.047619 23450 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:40:39.057384 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45931
--builtin_ntp_servers=127.8.241.212:45983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 18:40:39.177129 23459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:39.177377 23459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:39.177429 23459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:39.181344 23459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:39.181393 23459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:39.181463 23459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:39.186447 23459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:45983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:39.187445 23459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:39.188779 23459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:39.198913 23468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:39.199369 23467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:39.199946 23470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:39.202147 23459 server_base.cc:1034] running on GCE node
I20260806 18:40:39.202613 23459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:39.203382 23459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:39.204602 23459 hybrid_clock.cc:648] HybridClock initialized: now 1786041639204568 us; error 52 us; skew 500 ppm
I20260806 18:40:39.206799 23459 webserver.cc:460] Webserver started at http://127.8.241.193:38073/ using document root <none> and password file <none>
I20260806 18:40:39.207372 23459 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:39.207435 23459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:39.207623 23459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:39.209767 23459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "6d36b1a0246f4c9dbcce3e581686862b"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "5076705cb3349c48eed4cb772b614514"
server_key_iv: "6971829e2360afd5c0dacb4a854d71ec"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.210310 23459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "6d36b1a0246f4c9dbcce3e581686862b"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "5076705cb3349c48eed4cb772b614514"
server_key_iv: "6971829e2360afd5c0dacb4a854d71ec"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.214113 23459 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:40:39.216666 23483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.218004 23459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:40:39.218112 23459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "6d36b1a0246f4c9dbcce3e581686862b"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "5076705cb3349c48eed4cb772b614514"
server_key_iv: "6971829e2360afd5c0dacb4a854d71ec"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.218207 23459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:39.237191 23459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:39.237865 23459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:39.238047 23459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:39.238622 23459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:39.239573 23459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:39.239620 23459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.239673 23459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:39.239701 23459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.249908 23459 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:36847
I20260806 18:40:39.249926 23642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:36847 every 8 connection(s)
I20260806 18:40:39.250944 23459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:39.254624 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 23459
I20260806 18:40:39.254736 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:39.255064 9159 external_mini_cluster.cc:1428] Setting key 7a5c5a76991eb662c4fee15d014b6f3e
I20260806 18:40:39.256979 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45931
--builtin_ntp_servers=127.8.241.212:45983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:40:39.261814 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:39.252832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:34461 (local address 127.8.241.254:45931)
0806 18:40:39.252991 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:39.252997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:39.254275 (+ 1278us) server_negotiation.cc:407] Connection header received
0806 18:40:39.256271 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:39.256276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:39.256342 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:39.256401 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:39.257924 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.258656 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.259852 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.260081 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.260171 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:39.261320 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:39.261323 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:39.261326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:39.261341 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:39.261360 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:39.261368 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:39.261429 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:39.261593 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 18:40:39.261663 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:40:39.262785 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:39.253256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45931 (local address 127.8.241.193:34461)
0806 18:40:39.254097 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:39.254150 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:39.255982 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:39.256874 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:39.256883 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:39.257052 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:39.257747 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:39.257778 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.258800 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.258803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:39.259718 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:39.259728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.260233 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.260238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:39.260418 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:39.260995 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:39.260997 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:39.261080 (+ 83us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:39.261157 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:39.261439 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:39.261450 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:39.261456 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:40:39.261576 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 18:40:39.261906 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":150,"threads_started":1}
I20260806 18:40:39.264037 23643 heartbeater.cc:347] Connected to a master server at 127.8.241.254:45931
I20260806 18:40:39.264294 23643 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:39.264861 23643 heartbeater.cc:511] Master 127.8.241.254:45931 requested a full tablet report, sending...
I20260806 18:40:39.266472 23373 ts_manager.cc:194] Registered new tserver with Master: 6d36b1a0246f4c9dbcce3e581686862b (127.8.241.193:36847)
I20260806 18:40:39.267648 23373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.193:34461
W20260806 18:40:39.408339 23647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:39.408587 23647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:39.408622 23647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:39.412554 23647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:39.412616 23647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:39.412667 23647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:39.417332 23647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:45983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:39.418321 23647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:39.419623 23647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:39.428982 23657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:39.428987 23658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:39.428994 23662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:39.429361 23647 server_base.cc:1034] running on GCE node
I20260806 18:40:39.429765 23647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:39.430418 23647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:39.431610 23647 hybrid_clock.cc:648] HybridClock initialized: now 1786041639431588 us; error 33 us; skew 500 ppm
I20260806 18:40:39.433573 23647 webserver.cc:460] Webserver started at http://127.8.241.194:34919/ using document root <none> and password file <none>
I20260806 18:40:39.434168 23647 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:39.434237 23647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:39.434438 23647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:39.436542 23647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "0dfda2537cb64f278ffa65d4e099c3be"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "77b34a302176f4f7d36dad8411b05a71"
server_key_iv: "d9d0ee75bb8f2e618f1e243e915296ae"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.437144 23647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "0dfda2537cb64f278ffa65d4e099c3be"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "77b34a302176f4f7d36dad8411b05a71"
server_key_iv: "d9d0ee75bb8f2e618f1e243e915296ae"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.441174 23647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:40:39.443763 23674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.445091 23647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:39.445205 23647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "0dfda2537cb64f278ffa65d4e099c3be"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "77b34a302176f4f7d36dad8411b05a71"
server_key_iv: "d9d0ee75bb8f2e618f1e243e915296ae"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.445307 23647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:39.461789 23647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:39.462489 23647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:39.462667 23647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:39.463274 23647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:39.464280 23647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:39.464329 23647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.464385 23647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:39.464413 23647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.472991 23647 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:33959
I20260806 18:40:39.473057 23839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:33959 every 8 connection(s)
I20260806 18:40:39.473930 23647 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:39.474720 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 23647
I20260806 18:40:39.474838 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:40:39.475183 9159 external_mini_cluster.cc:1428] Setting key 5d99601a0b5cdeddf94787ae3b9a705b
I20260806 18:40:39.477242 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45931
--builtin_ntp_servers=127.8.241.212:45983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:40:39.485306 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:39.475852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:57465 (local address 127.8.241.254:45931)
0806 18:40:39.476010 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:39.476014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:39.477417 (+ 1403us) server_negotiation.cc:407] Connection header received
0806 18:40:39.479234 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:39.479238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:39.479295 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:39.479382 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:39.480539 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.481342 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.482954 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.483300 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.483418 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:39.484743 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:39.484747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:39.484749 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:39.484765 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:39.484788 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:39.484798 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:39.484872 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:39.485091 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 18:40:39.485154 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:40:39.486490 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:39.476334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45931 (local address 127.8.241.194:57465)
0806 18:40:39.477252 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:39.477323 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:39.478993 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:39.479585 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:39.479595 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:39.479701 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:39.480360 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:39.480380 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.481476 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.481480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:39.482804 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:39.482816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.483482 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.483488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:39.483706 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:39.484414 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:39.484416 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:39.484475 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:39.484565 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:39.484888 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:39.484897 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:39.484904 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:40:39.485094 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 18:40:39.485420 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":121,"threads_started":1}
I20260806 18:40:39.487784 23840 heartbeater.cc:347] Connected to a master server at 127.8.241.254:45931
I20260806 18:40:39.488060 23840 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:39.488735 23840 heartbeater.cc:511] Master 127.8.241.254:45931 requested a full tablet report, sending...
I20260806 18:40:39.490061 23373 ts_manager.cc:194] Registered new tserver with Master: 0dfda2537cb64f278ffa65d4e099c3be (127.8.241.194:33959)
I20260806 18:40:39.490828 23373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.194:57465
W20260806 18:40:39.596467 23846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:39.596719 23846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:39.596752 23846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:39.600667 23846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:39.600726 23846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:39.600777 23846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:39.605715 23846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:45983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:45931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:39.606765 23846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:39.608193 23846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:39.617725 23852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:39.617785 23854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:39.617738 23857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:39.617897 23846 server_base.cc:1034] running on GCE node
I20260806 18:40:39.618324 23846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:39.618932 23846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:39.620090 23846 hybrid_clock.cc:648] HybridClock initialized: now 1786041639620075 us; error 27 us; skew 500 ppm
I20260806 18:40:39.621903 23846 webserver.cc:460] Webserver started at http://127.8.241.195:40605/ using document root <none> and password file <none>
I20260806 18:40:39.622473 23846 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:39.622540 23846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:39.622720 23846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:39.624708 23846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "85dba24ddfdb4f548c282acb31633739"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "03573d3d59fc402b66730794bcd16b1d"
server_key_iv: "81a6630dfde029b2235a33658f2a86db"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.625329 23846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "85dba24ddfdb4f548c282acb31633739"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "03573d3d59fc402b66730794bcd16b1d"
server_key_iv: "81a6630dfde029b2235a33658f2a86db"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.628996 23846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 18:40:39.631572 23869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.632759 23846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:39.632858 23846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "85dba24ddfdb4f548c282acb31633739"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "03573d3d59fc402b66730794bcd16b1d"
server_key_iv: "81a6630dfde029b2235a33658f2a86db"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.632951 23846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:39.656195 23846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:39.657152 23846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:39.657348 23846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:39.657991 23846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:39.659000 23846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:39.659051 23846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.659107 23846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:39.659137 23846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.669354 23846 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:41181
I20260806 18:40:39.669387 24026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:41181 every 8 connection(s)
I20260806 18:40:39.670315 23846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:39.676764 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 23846
I20260806 18:40:39.676872 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:39.677204 9159 external_mini_cluster.cc:1428] Setting key 297d171773d66a014c592dbe96fb4137
I20260806 18:40:39.681490 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:39.672249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:41289 (local address 127.8.241.254:45931)
0806 18:40:39.672397 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:39.672403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:39.673764 (+ 1361us) server_negotiation.cc:407] Connection header received
0806 18:40:39.675691 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:39.675693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:39.675742 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:39.675817 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:39.677094 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.677790 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.679502 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.679722 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.679803 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:39.681045 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:39.681049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:39.681052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:39.681066 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:39.681086 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:39.681097 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:39.681153 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:39.681314 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 18:40:39.681365 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:40:39.682693 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:39.672690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:45931 (local address 127.8.241.195:41289)
0806 18:40:39.673593 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:39.673652 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:39.675438 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:39.676023 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:39.676033 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:39.676150 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:39.676895 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:39.676914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.677935 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.677940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:39.679374 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:39.679387 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:39.679870 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:39.679874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:39.680081 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:39.680710 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:39.680713 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:39.680774 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:39.680868 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:39.681173 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:39.681181 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:39.681188 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:40:39.681322 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 18:40:39.681651 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":123,"threads_started":1}
I20260806 18:40:39.683848 24027 heartbeater.cc:347] Connected to a master server at 127.8.241.254:45931
I20260806 18:40:39.684088 24027 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:39.684615 24027 heartbeater.cc:511] Master 127.8.241.254:45931 requested a full tablet report, sending...
I20260806 18:40:39.685658 23373 ts_manager.cc:194] Registered new tserver with Master: 85dba24ddfdb4f548c282acb31633739 (127.8.241.195:41181)
I20260806 18:40:39.686199 23373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.195:41289
I20260806 18:40:39.690814 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:40:39.693660 23430 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:39.692742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.8.241.254:45931)
0806 18:40:39.692877 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:39.692882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:39.693020 (+ 138us) server_negotiation.cc:407] Connection header received
0806 18:40:39.693148 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:39.693152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:39.693204 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:39.693289 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:39.693523 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58182: BlockingRecv error: recv got EOF from 127.0.0.1:58182 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:40:39.693729 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 23459
I20260806 18:40:39.717602 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 23647
I20260806 18:40:39.740303 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 23846
I20260806 18:40:39.752661 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 23328
2026-08-06T18:40:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (969 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T18:40:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:39Z Disabled control of system clock
I20260806 18:40:39.788148 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34585
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:35843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:34585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:40:39.899950 24045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:39.900223 24045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:39.900262 24045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:39.904326 24045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:39.904388 24045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:39.904403 24045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:39.904421 24045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:39.904458 24045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:39.909219 24045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:35843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:34585
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:39.910315 24045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:39.911755 24045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:39.919955 24053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:39.920009 24055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:39.920207 24045 server_base.cc:1034] running on GCE node
W20260806 18:40:39.919955 24057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:39.920819 24045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:39.921944 24045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:39.923113 24045 hybrid_clock.cc:648] HybridClock initialized: now 1786041639923093 us; error 27 us; skew 500 ppm
I20260806 18:40:39.924994 24045 webserver.cc:460] Webserver started at http://127.8.241.254:39901/ using document root <none> and password file <none>
I20260806 18:40:39.925640 24045 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:39.925712 24045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:39.925941 24045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:39.928056 24045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "8e746283d2d54839b5a70c75a7b4fb35"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "10d8dd00a8f7a68461bdedbcd9d47232"
server_key_iv: "83739c82856d0124fb1a2154e93003a6"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.928642 24045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "8e746283d2d54839b5a70c75a7b4fb35"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "10d8dd00a8f7a68461bdedbcd9d47232"
server_key_iv: "83739c82856d0124fb1a2154e93003a6"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.932453 24045 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260806 18:40:39.935014 24069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:39.936199 24045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:39.936327 24045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "8e746283d2d54839b5a70c75a7b4fb35"
format_stamp: "Formatted at 2026-08-06 18:40:39 on dist-test-slave-k5rr"
server_key: "10d8dd00a8f7a68461bdedbcd9d47232"
server_key_iv: "83739c82856d0124fb1a2154e93003a6"
server_key_version: "encryptionkey@0"
I20260806 18:40:39.936425 24045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:39.941084 24045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:39.941696 24045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:39.941875 24045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:39.948406 24045 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:34585
I20260806 18:40:39.948422 24146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:34585 every 8 connection(s)
I20260806 18:40:39.949373 24045 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:39.952620 24149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:39.953737 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 24045
I20260806 18:40:39.953836 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:39.954114 9159 external_mini_cluster.cc:1428] Setting key 3af2f72a82dd8cae4b97c796f3fe5818
I20260806 18:40:39.958770 24149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35: Bootstrap starting.
I20260806 18:40:39.959507 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:39.955503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43896 (local address 127.8.241.254:34585)
0806 18:40:39.956112 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:39.956132 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:39.956167 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:40:39.956776 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:39.956792 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:39.957180 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:39.957454 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:39.958270 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:39.958275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:39.958287 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:39.958297 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:39.958352 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:39.958368 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:39.958471 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:39.958570 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 18:40:39.958806 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":117,"threads_started":1}
I20260806 18:40:39.961278 24149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:39.962142 24149 log.cc:824] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:39.963961 24149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35: No bootstrap required, opened a new log
I20260806 18:40:39.967697 24149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e746283d2d54839b5a70c75a7b4fb35" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34585 } }
I20260806 18:40:39.967924 24149 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:39.967978 24149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e746283d2d54839b5a70c75a7b4fb35, State: Initialized, Role: FOLLOWER
I20260806 18:40:39.968348 24149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e746283d2d54839b5a70c75a7b4fb35" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34585 } }
I20260806 18:40:39.968469 24149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:39.968514 24149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:39.968585 24149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:39.969481 24149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e746283d2d54839b5a70c75a7b4fb35" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34585 } }
I20260806 18:40:39.969785 24149 leader_election.cc:302] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e746283d2d54839b5a70c75a7b4fb35; no voters:
I20260806 18:40:39.970052 24149 leader_election.cc:288] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:39.970144 24155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:39.970371 24155 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [term 1 LEADER]: Becoming Leader. State: Replica: 8e746283d2d54839b5a70c75a7b4fb35, State: Running, Role: LEADER
I20260806 18:40:39.970755 24155 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e746283d2d54839b5a70c75a7b4fb35" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34585 } }
I20260806 18:40:39.971294 24149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:39.972329 24156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e746283d2d54839b5a70c75a7b4fb35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e746283d2d54839b5a70c75a7b4fb35" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34585 } } }
I20260806 18:40:39.972469 24156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:39.972756 24157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e746283d2d54839b5a70c75a7b4fb35. Latest consensus state: current_term: 1 leader_uuid: "8e746283d2d54839b5a70c75a7b4fb35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e746283d2d54839b5a70c75a7b4fb35" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34585 } } }
I20260806 18:40:39.972888 24157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35 [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:39.973071 24168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:39.976135 24168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:39.982014 24168 catalog_manager.cc:1347] Generated new cluster ID: 8be5abd45c6b42fa950f28b92815bb5e
I20260806 18:40:39.982075 24168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:39.992964 24168 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:39.993325 24168 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:40.012578 24168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e746283d2d54839b5a70c75a7b4fb35: Generated new TSK 0
I20260806 18:40:40.013547 24168 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:40:40.016364 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34585
--builtin_ntp_servers=127.8.241.212:35843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:40:40.130386 24186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:40.130651 24186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:40.130707 24186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:40.134651 24186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:40.134704 24186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:40.134783 24186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:40.139462 24186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:35843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:40.140779 24186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:40.142313 24186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:40.151818 24198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:40.151882 24194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:40.151983 24186 server_base.cc:1034] running on GCE node
W20260806 18:40:40.151813 24196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:40.152491 24186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:40.153146 24186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:40.154320 24186 hybrid_clock.cc:648] HybridClock initialized: now 1786041640154295 us; error 32 us; skew 500 ppm
I20260806 18:40:40.156064 24186 webserver.cc:460] Webserver started at http://127.8.241.193:44045/ using document root <none> and password file <none>
I20260806 18:40:40.156626 24186 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:40.156688 24186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:40.156893 24186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:40.158946 24186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "f459da08affd4c3ba254ae04b312f5e4"
format_stamp: "Formatted at 2026-08-06 18:40:40 on dist-test-slave-k5rr"
server_key: "91aeac17502d2e59000cd15706899ac8"
server_key_iv: "2d1fe1956d90f1ef0461156699c7e8be"
server_key_version: "encryptionkey@0"
I20260806 18:40:40.159472 24186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "f459da08affd4c3ba254ae04b312f5e4"
format_stamp: "Formatted at 2026-08-06 18:40:40 on dist-test-slave-k5rr"
server_key: "91aeac17502d2e59000cd15706899ac8"
server_key_iv: "2d1fe1956d90f1ef0461156699c7e8be"
server_key_version: "encryptionkey@0"
I20260806 18:40:40.163055 24186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:40:40.165581 24209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:40.166793 24186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:40.166911 24186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "f459da08affd4c3ba254ae04b312f5e4"
format_stamp: "Formatted at 2026-08-06 18:40:40 on dist-test-slave-k5rr"
server_key: "91aeac17502d2e59000cd15706899ac8"
server_key_iv: "2d1fe1956d90f1ef0461156699c7e8be"
server_key_version: "encryptionkey@0"
I20260806 18:40:40.167021 24186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:40.171609 24186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:40.172210 24186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:40.172367 24186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:40.172942 24186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:40.173921 24186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:40.173970 24186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:40.174014 24186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:40.174042 24186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:40.182403 24186 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:43807
I20260806 18:40:40.182430 24388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:43807 every 8 connection(s)
I20260806 18:40:40.183255 24186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:40.188879 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:40.185027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:35377 (local address 127.8.241.254:34585)
0806 18:40:40.185167 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:40.185171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:40.186207 (+ 1036us) server_negotiation.cc:407] Connection header received
0806 18:40:40.187469 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:40.187472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:40.187508 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:40.187578 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:40.188571 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:40.188574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:40.188576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:40.188582 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:40.188595 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:40.188601 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:40.188657 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:40.188742 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 18:40:40.188776 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:40:40.189713 24392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:40.185430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34585 (local address 127.8.241.193:35377)
0806 18:40:40.186070 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:40.186111 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:40.187292 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:40.187721 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:40.187728 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:40.187811 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:40.188347 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:40.188348 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:40.188389 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:40.188461 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:40.188650 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:40.188654 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:40.188658 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:40:40.188775 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 18:40:40.188984 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":102,"threads_started":1}
I20260806 18:40:40.190837 24390 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34585
I20260806 18:40:40.191054 24390 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:40.191435 24390 heartbeater.cc:511] Master 127.8.241.254:34585 requested a full tablet report, sending...
I20260806 18:40:40.192878 24095 ts_manager.cc:194] Registered new tserver with Master: f459da08affd4c3ba254ae04b312f5e4 (127.8.241.193:43807)
I20260806 18:40:40.193300 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 24186
I20260806 18:40:40.193415 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:40.193754 9159 external_mini_cluster.cc:1428] Setting key bb84863d7a0704732a26fb7d2ca3b0e2
I20260806 18:40:40.195535 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34585
--builtin_ntp_servers=127.8.241.212:35843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:40:40.310256 24395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:40.310515 24395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:40.310554 24395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:40.314702 24395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:40.314771 24395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:40.314828 24395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:40.319520 24395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:35843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:40.320585 24395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:40.322021 24395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:40:40.331583 24395 server_base.cc:1034] running on GCE node
W20260806 18:40:40.331523 24401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:40.331519 24404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:40.331523 24402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:40.332221 24395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:40.332834 24395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:40.333995 24395 hybrid_clock.cc:648] HybridClock initialized: now 1786041640333980 us; error 53 us; skew 500 ppm
I20260806 18:40:40.335788 24395 webserver.cc:460] Webserver started at http://127.8.241.194:41901/ using document root <none> and password file <none>
I20260806 18:40:40.336500 24395 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:40.336591 24395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:40.336809 24395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:40.338976 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "c71ba35737cc44fe9e4bbd64ea01a880"
format_stamp: "Formatted at 2026-08-06 18:40:40 on dist-test-slave-k5rr"
server_key: "ae3b57a5a43a5c179f2cd59eab2c0624"
server_key_iv: "7a106d8bb610b7f83a63db2deb1194d6"
server_key_version: "encryptionkey@0"
I20260806 18:40:40.339569 24395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "c71ba35737cc44fe9e4bbd64ea01a880"
format_stamp: "Formatted at 2026-08-06 18:40:40 on dist-test-slave-k5rr"
server_key: "ae3b57a5a43a5c179f2cd59eab2c0624"
server_key_iv: "7a106d8bb610b7f83a63db2deb1194d6"
server_key_version: "encryptionkey@0"
I20260806 18:40:40.343374 24395 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:40:40.345894 24413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:40.347200 24395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:40.347309 24395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "c71ba35737cc44fe9e4bbd64ea01a880"
format_stamp: "Formatted at 2026-08-06 18:40:40 on dist-test-slave-k5rr"
server_key: "ae3b57a5a43a5c179f2cd59eab2c0624"
server_key_iv: "7a106d8bb610b7f83a63db2deb1194d6"
server_key_version: "encryptionkey@0"
I20260806 18:40:40.347412 24395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:40.352164 24395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:40.352759 24395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:40.352913 24395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:40.353469 24395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:40.354382 24395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:40.354429 24395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:40.354477 24395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:40.354502 24395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:40.362625 24395 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:41687
I20260806 18:40:40.362665 24574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:41687 every 8 connection(s)
I20260806 18:40:40.363521 24395 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:40.369395 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:40.365290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:54989 (local address 127.8.241.254:34585)
0806 18:40:40.365449 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:40.365453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:40.366498 (+ 1045us) server_negotiation.cc:407] Connection header received
0806 18:40:40.367783 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:40.367786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:40.367826 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:40.367901 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:40.368930 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:40.368933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:40.368935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:40.368947 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:40.368968 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:40.368978 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:40.369109 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:40.369252 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:40:40.369290 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:40:40.370214 24580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:40.365701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34585 (local address 127.8.241.194:54989)
0806 18:40:40.366359 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:40.366398 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:40.367581 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:40.368060 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:40.368067 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:40.368153 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:40.368675 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:40.368677 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:40.368715 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:40.368786 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:40.369104 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:40.369114 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:40.369119 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:40:40.369275 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 18:40:40.369461 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
I20260806 18:40:40.371323 24578 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34585
I20260806 18:40:40.371565 24578 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:40.372092 24578 heartbeater.cc:511] Master 127.8.241.254:34585 requested a full tablet report, sending...
I20260806 18:40:40.372427 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 24395
I20260806 18:40:40.372563 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:40:40.372871 9159 external_mini_cluster.cc:1428] Setting key 84117d8f8e10763db506ffb481062c0e
I20260806 18:40:40.373258 24095 ts_manager.cc:194] Registered new tserver with Master: c71ba35737cc44fe9e4bbd64ea01a880 (127.8.241.194:41687)
I20260806 18:40:40.374922 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34585
--builtin_ntp_servers=127.8.241.212:35843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:40:40.487025 24581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:40.487286 24581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:40.487326 24581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:40.491318 24581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:40.491370 24581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:40.491441 24581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:40.496145 24581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:35843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:40.497241 24581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:40.498636 24581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:40.508122 24591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:40.508181 24589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:40.508128 24588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:40.508363 24581 server_base.cc:1034] running on GCE node
I20260806 18:40:40.508833 24581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:40.509498 24581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:40.510655 24581 hybrid_clock.cc:648] HybridClock initialized: now 1786041640510635 us; error 31 us; skew 500 ppm
I20260806 18:40:40.512462 24581 webserver.cc:460] Webserver started at http://127.8.241.195:33983/ using document root <none> and password file <none>
I20260806 18:40:40.512998 24581 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:40.513103 24581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:40.513283 24581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:40.515290 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "656eb8dade72457db05030f2e189f0fb"
format_stamp: "Formatted at 2026-08-06 18:40:40 on dist-test-slave-k5rr"
server_key: "4b7876d6bb65b5330ed96f985d8ce555"
server_key_iv: "2dff3b329b48011a7a93e087474fde05"
server_key_version: "encryptionkey@0"
I20260806 18:40:40.515799 24581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "656eb8dade72457db05030f2e189f0fb"
format_stamp: "Formatted at 2026-08-06 18:40:40 on dist-test-slave-k5rr"
server_key: "4b7876d6bb65b5330ed96f985d8ce555"
server_key_iv: "2dff3b329b48011a7a93e087474fde05"
server_key_version: "encryptionkey@0"
I20260806 18:40:40.519428 24581 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:40:40.521858 24605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:40.523039 24581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:40.523136 24581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "656eb8dade72457db05030f2e189f0fb"
format_stamp: "Formatted at 2026-08-06 18:40:40 on dist-test-slave-k5rr"
server_key: "4b7876d6bb65b5330ed96f985d8ce555"
server_key_iv: "2dff3b329b48011a7a93e087474fde05"
server_key_version: "encryptionkey@0"
I20260806 18:40:40.523224 24581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:40.527652 24581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:40.528247 24581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:40.528400 24581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:40.528940 24581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:40.530061 24581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:40.530110 24581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:40.530154 24581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:40.530184 24581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:40.538481 24581 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:45501
I20260806 18:40:40.538532 24779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:45501 every 8 connection(s)
I20260806 18:40:40.539382 24581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:40.541030 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 24581
I20260806 18:40:40.541136 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:40.541438 9159 external_mini_cluster.cc:1428] Setting key 61525cfc914f9f1924f345b277a6cf7f
I20260806 18:40:40.545572 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:40.541190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:59981 (local address 127.8.241.254:34585)
0806 18:40:40.541363 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:40.541368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:40.542621 (+ 1253us) server_negotiation.cc:407] Connection header received
0806 18:40:40.544060 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:40.544063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:40.544100 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:40.544165 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:40.545241 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:40.545244 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:40.545246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:40:40.545256 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:40.545274 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:40:40.545281 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:40.545341 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:40.545442 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 18:40:40.545476 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 18:40:40.546406 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:40.541676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34585 (local address 127.8.241.195:59981)
0806 18:40:40.542467 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:40.542522 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:40.543878 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:40.544309 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:40.544315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:40.544407 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:40.544940 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:40:40.544942 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:40.544983 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:40:40.545110 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:40.545339 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:40.545343 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:40.545347 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:40:40.545469 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 18:40:40.545656 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":139,"threads_started":1}
I20260806 18:40:40.547371 24780 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34585
I20260806 18:40:40.547578 24780 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:40.547974 24780 heartbeater.cc:511] Master 127.8.241.254:34585 requested a full tablet report, sending...
I20260806 18:40:40.548945 24094 ts_manager.cc:194] Registered new tserver with Master: 656eb8dade72457db05030f2e189f0fb (127.8.241.195:45501)
I20260806 18:40:40.555140 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:40:40.557799 24153 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:40:40.556974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43900 (local address 127.8.241.254:34585)
0806 18:40:40.557115 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:40.557120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:40.557265 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:40:40.557360 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:40.557362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:40.557396 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:40.557465 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:40.557676 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43900: BlockingRecv error: recv got EOF from 127.0.0.1:43900 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:40:40.557936 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 24186
I20260806 18:40:40.582070 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 24395
I20260806 18:40:40.592113 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 24581
I20260806 18:40:40.603744 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 24045
2026-08-06T18:40:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (854 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:40:40 dist-test-slave-k5rr krb5kdc[24791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:40:40 dist-test-slave-k5rr krb5kdc[24791](info): set up 2 sockets
Aug 06 18:40:40 dist-test-slave-k5rr krb5kdc[24791](info): commencing operation
krb5kdc: starting...
W20260806 18:40:41.753217 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:40:41 dist-test-slave-k5rr krb5kdc[24791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:40:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:40:41.872246 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34797
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:34797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:41.983484 24810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:41.983760 24810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:41.983817 24810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:41.987773 24810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:40:41.987833 24810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:41.987859 24810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:41.987879 24810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:40:41.987900 24810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:40:41.993407 24810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:34797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:41.994467 24810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:41.995501 24810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:42.003834 24823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:42.003850 24818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:42.003858 24819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:42.004082 24810 server_base.cc:1034] running on GCE node
I20260806 18:40:42.004724 24810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:42.005827 24810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:42.007001 24810 hybrid_clock.cc:648] HybridClock initialized: now 1786041642006983 us; error 31 us; skew 500 ppm
Aug 06 18:40:42 dist-test-slave-k5rr krb5kdc[24791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041642, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:42.013700 24810 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:40:42.014644 24810 webserver.cc:460] Webserver started at http://127.8.241.254:42543/ using document root <none> and password file <none>
I20260806 18:40:42.015198 24810 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:42.015264 24810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:42.015480 24810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:42.017633 24810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "5024e84c99dc4b4c82c53957e07215ea"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "3905c528473ec993c1784d10efc711f3"
server_key_iv: "d3fdbf1cb6abcb945da44e92bb156e7d"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.018179 24810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "5024e84c99dc4b4c82c53957e07215ea"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "3905c528473ec993c1784d10efc711f3"
server_key_iv: "d3fdbf1cb6abcb945da44e92bb156e7d"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.021864 24810 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:40:42.024288 24837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.025579 24810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:40:42.025704 24810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "5024e84c99dc4b4c82c53957e07215ea"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "3905c528473ec993c1784d10efc711f3"
server_key_iv: "d3fdbf1cb6abcb945da44e92bb156e7d"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.025808 24810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:42.050127 24810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:42.052964 24810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:42.053206 24810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:42.060321 24810 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:34797
I20260806 18:40:42.060598 24919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:34797 every 8 connection(s)
I20260806 18:40:42.061396 24810 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:40:42.064286 24921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:42.068310 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 24810
I20260806 18:40:42.068434 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:40:42.068773 9159 external_mini_cluster.cc:1428] Setting key 132fef026d14e3b9eb52673ac5ed3bd9
I20260806 18:40:42.070648 24921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea: Bootstrap starting.
I20260806 18:40:42.073252 24921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:42.073982 24921 log.cc:824] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:42.075911 24921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea: No bootstrap required, opened a new log
Aug 06 18:40:42 dist-test-slave-k5rr krb5kdc[24791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:42.079661 24921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5024e84c99dc4b4c82c53957e07215ea" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34797 } }
I20260806 18:40:42.079895 24921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:42.079955 24921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5024e84c99dc4b4c82c53957e07215ea, State: Initialized, Role: FOLLOWER
I20260806 18:40:42.080490 24921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5024e84c99dc4b4c82c53957e07215ea" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34797 } }
I20260806 18:40:42.080680 24921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:40:42.080752 24921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:40:42.080848 24921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:42.082000 24921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5024e84c99dc4b4c82c53957e07215ea" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34797 } }
I20260806 18:40:42.082474 24921 leader_election.cc:302] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5024e84c99dc4b4c82c53957e07215ea; no voters:
I20260806 18:40:42.082849 24921 leader_election.cc:288] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:40:42.082888 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.070225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.8.241.254:34797)
0806 18:40:42.070647 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.070663 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.070700 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:40:42.071381 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.071401 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.071758 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.072050 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:42.072916 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.074014 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.074865 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.075337 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.075485 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.077922 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:42.077927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:42.077949 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:42.077978 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:42.079607 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.080037 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.080044 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.080050 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.080200 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.080474 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.080477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.080479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.081097 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:42.081311 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.081527 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.081842 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":125,"threads_started":1}
I20260806 18:40:42.082945 24928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:42.083182 24928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [term 1 LEADER]: Becoming Leader. State: Replica: 5024e84c99dc4b4c82c53957e07215ea, State: Running, Role: LEADER
I20260806 18:40:42.083550 24928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5024e84c99dc4b4c82c53957e07215ea" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34797 } }
I20260806 18:40:42.084189 24921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:40:42.085553 24929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5024e84c99dc4b4c82c53957e07215ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5024e84c99dc4b4c82c53957e07215ea" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34797 } } }
I20260806 18:40:42.085685 24929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:42.085800 24931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5024e84c99dc4b4c82c53957e07215ea. Latest consensus state: current_term: 1 leader_uuid: "5024e84c99dc4b4c82c53957e07215ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5024e84c99dc4b4c82c53957e07215ea" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34797 } } }
I20260806 18:40:42.085901 24931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea [sys.catalog]: This master's current role is: LEADER
I20260806 18:40:42.086263 24941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:40:42.089711 24941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:40:42.095772 24941 catalog_manager.cc:1347] Generated new cluster ID: 437fdbe249754ac69c68b1681dd94721
I20260806 18:40:42.095860 24941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:40:42.112651 24941 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:40:42.114085 24941 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:40:42.118953 24941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea: Generated new TSK 0
I20260806 18:40:42.119812 24941 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:40:42.149221 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34797
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:42.265193 24968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:42.265470 24968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:42.265530 24968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:42.269523 24968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:42.269588 24968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:42.269666 24968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:40:42.274997 24968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:42.276109 24968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:42.277242 24968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:42.287137 24974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:42.287114 24975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:42.287114 24978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:42.287336 24968 server_base.cc:1034] running on GCE node
I20260806 18:40:42.287834 24968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:42.288482 24968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:42.289635 24968 hybrid_clock.cc:648] HybridClock initialized: now 1786041642289606 us; error 43 us; skew 500 ppm
Aug 06 18:40:42 dist-test-slave-k5rr krb5kdc[24791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041642, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:42.292408 24968 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:40:42.293387 24968 webserver.cc:460] Webserver started at http://127.8.241.193:33703/ using document root <none> and password file <none>
I20260806 18:40:42.293965 24968 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:42.294039 24968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:42.294257 24968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:42.296300 24968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "75ff436ca3e04e2b81c611beefd6083a"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "d52369a33f8b86f343e1211c7be27c84"
server_key_iv: "245a243eaca36c390c0dc0fd9e892727"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.296861 24968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "75ff436ca3e04e2b81c611beefd6083a"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "d52369a33f8b86f343e1211c7be27c84"
server_key_iv: "245a243eaca36c390c0dc0fd9e892727"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.300518 24968 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:40:42.302975 24990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.304167 24968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:42.304279 24968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "75ff436ca3e04e2b81c611beefd6083a"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "d52369a33f8b86f343e1211c7be27c84"
server_key_iv: "245a243eaca36c390c0dc0fd9e892727"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.304375 24968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:42.323521 24968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:42.326663 24968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:42.326818 24968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:42.327387 24968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:42.328208 24968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:42.328258 24968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.328297 24968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:42.328317 24968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.336879 24968 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:32975
I20260806 18:40:42.336900 25169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:32975 every 8 connection(s)
I20260806 18:40:42.337807 24968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:40:42.346462 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 24968
I20260806 18:40:42.346596 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:40:42.346884 9159 external_mini_cluster.cc:1428] Setting key ff09438915a1acd969cb0b3651c856ae
Aug 06 18:40:42 dist-test-slave-k5rr krb5kdc[24791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041642, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:42.351385 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.339676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:37269 (local address 127.8.241.254:34797)
0806 18:40:42.339822 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.339827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.340726 (+ 899us) server_negotiation.cc:407] Connection header received
0806 18:40:42.342092 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.342095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.342143 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.342217 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:42.343353 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.343878 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.344910 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.345194 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.345270 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.347759 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:42.347763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:42.347766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:42.347791 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:42.349271 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.349826 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.349829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.349830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.349899 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.350278 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.350282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.350283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.350519 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:42.350618 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.351098 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.351227 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:40:42.352178 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.340093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34797 (local address 127.8.241.193:37269)
0806 18:40:42.340566 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:42.340612 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:42.341882 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:42.342368 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:42.342375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:42.342735 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:42.343236 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.343250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.343974 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.343976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.344821 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.344830 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.345315 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.345319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.345459 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.345964 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:42.345981 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:42.347591 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:42.349391 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:42.349398 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:42.349415 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:42.349742 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:42.349973 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:42.349976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:42.349978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:42.350166 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:42.350613 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:42.350619 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:42.350866 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 18:40:42.351034 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 18:40:42.351291 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":122,"threads_started":1}
I20260806 18:40:42.353430 25172 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34797
I20260806 18:40:42.353679 25172 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:42.354179 25172 heartbeater.cc:511] Master 127.8.241.254:34797 requested a full tablet report, sending...
I20260806 18:40:42.355892 24863 ts_manager.cc:194] Registered new tserver with Master: 75ff436ca3e04e2b81c611beefd6083a (127.8.241.193:32975)
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:40:42.357538 24863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:37269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:40:42.373728 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34797
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:42.486409 25179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:42.486651 25179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:42.486707 25179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:42.490615 25179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:42.490669 25179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:42.490723 25179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:40:42.495743 25179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:42.496704 25179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:42.497692 25179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:42.507191 25193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:42.507159 25187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:42.507226 25188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:42.507412 25179 server_base.cc:1034] running on GCE node
I20260806 18:40:42.507813 25179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:42.508435 25179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:42.509603 25179 hybrid_clock.cc:648] HybridClock initialized: now 1786041642509578 us; error 36 us; skew 500 ppm
Aug 06 18:40:42 dist-test-slave-k5rr krb5kdc[24791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041642, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:42.512305 25179 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:40:42.513366 25179 webserver.cc:460] Webserver started at http://127.8.241.194:40333/ using document root <none> and password file <none>
I20260806 18:40:42.513965 25179 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:42.514029 25179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:42.514204 25179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:42.516260 25179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "7168f43bb4c86ba64f1891157cdb64fa"
server_key_iv: "2d87cc4f3fdd6e5593a5e19f126c864a"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.516780 25179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "7168f43bb4c86ba64f1891157cdb64fa"
server_key_iv: "2d87cc4f3fdd6e5593a5e19f126c864a"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.520412 25179 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:40:42.522783 25203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.524027 25179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:40:42.524122 25179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "7168f43bb4c86ba64f1891157cdb64fa"
server_key_iv: "2d87cc4f3fdd6e5593a5e19f126c864a"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.524204 25179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:42.534896 25179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:42.537933 25179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:42.538084 25179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:42.538607 25179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:42.539418 25179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:42.539479 25179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.539537 25179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:42.539561 25179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.547849 25179 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:45015
I20260806 18:40:42.547874 25383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:45015 every 8 connection(s)
I20260806 18:40:42.548739 25179 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:40:42.549793 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 25179
I20260806 18:40:42.549890 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:40:42.550177 9159 external_mini_cluster.cc:1428] Setting key 5b42de119ee2418c6532bb3f56f14ed0
Aug 06 18:40:42 dist-test-slave-k5rr krb5kdc[24791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041642, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
I20260806 18:40:42.563177 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.550589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:46045 (local address 127.8.241.254:34797)
0806 18:40:42.550740 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.550745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.551633 (+ 888us) server_negotiation.cc:407] Connection header received
0806 18:40:42.553025 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.553029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.553083 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.553165 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:42.554695 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.555209 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.556831 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.557097 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.557176 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.559939 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:42.559942 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:42.559944 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:42.559967 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:42.561358 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.561855 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.561858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.561860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.561941 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.562214 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.562216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.562217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.562386 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:42.562470 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.562892 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.563049 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:40:42.563992 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.551015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34797 (local address 127.8.241.194:46045)
0806 18:40:42.551493 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:42.551547 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:42.552797 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:42.553398 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:42.553407 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:42.553873 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:42.554540 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.554561 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.555339 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.555343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.556690 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.556704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.557252 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.557257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.557473 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.558050 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:42.558072 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:42.559765 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:42.561476 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:42.561481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:42.561493 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:42.561773 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:42.562021 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:42.562023 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:42.562024 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:42.562147 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:42.562472 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:42.562477 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:42.562705 (+ 228us) client_negotiation.cc:769] Sending connection context
0806 18:40:42.562850 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 18:40:42.563114 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":110,"threads_started":1}
I20260806 18:40:42.565212 25384 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34797
I20260806 18:40:42.565469 25384 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:42.565966 25384 heartbeater.cc:511] Master 127.8.241.254:34797 requested a full tablet report, sending...
I20260806 18:40:42.567086 24863 ts_manager.cc:194] Registered new tserver with Master: 8d2d8677043e48ab8ed9ebefa79f2a7e (127.8.241.194:45015)
I20260806 18:40:42.567886 24863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:46045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:40:42.578033 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34797
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:40:42.691262 25392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:40:42.691534 25392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:40:42.691582 25392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:40:42.695561 25392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:40:42.695621 25392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:40:42.695688 25392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:40:42.700820 25392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:40:42.701870 25392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:40:42.702889 25392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:40:42.712667 25399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:42.712718 25398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:40:42.712698 25401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:40:42.712770 25392 server_base.cc:1034] running on GCE node
I20260806 18:40:42.713299 25392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:40:42.713973 25392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:40:42.715127 25392 hybrid_clock.cc:648] HybridClock initialized: now 1786041642715101 us; error 46 us; skew 500 ppm
Aug 06 18:40:42 dist-test-slave-k5rr krb5kdc[24791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041642, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:40:42.717845 25392 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:40:42.718755 25392 webserver.cc:460] Webserver started at http://127.8.241.195:39859/ using document root <none> and password file <none>
I20260806 18:40:42.719321 25392 fs_manager.cc:359] Metadata directory not provided
I20260806 18:40:42.719390 25392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:40:42.719600 25392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:40:42.721761 25392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "2e105adccbe440feb65ae844fcbe44cf"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "e2f7d084079a3e30b7374464af5c0b64"
server_key_iv: "5ce9443777a4db4166711e96a0d6cc7d"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.722332 25392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "2e105adccbe440feb65ae844fcbe44cf"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "e2f7d084079a3e30b7374464af5c0b64"
server_key_iv: "5ce9443777a4db4166711e96a0d6cc7d"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.726140 25392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:40:42.728746 25410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.730065 25392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:40:42.730177 25392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "2e105adccbe440feb65ae844fcbe44cf"
format_stamp: "Formatted at 2026-08-06 18:40:42 on dist-test-slave-k5rr"
server_key: "e2f7d084079a3e30b7374464af5c0b64"
server_key_iv: "5ce9443777a4db4166711e96a0d6cc7d"
server_key_version: "encryptionkey@0"
I20260806 18:40:42.730275 25392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:40:42.752233 25392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:40:42.755205 25392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:40:42.755402 25392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:40:42.756021 25392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:40:42.756906 25392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:40:42.756955 25392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.757042 25392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:40:42.757072 25392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:40:42.765831 25392 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:39263
I20260806 18:40:42.765854 25590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:39263 every 8 connection(s)
I20260806 18:40:42.766848 25392 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:40:42.774608 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 25392
I20260806 18:40:42.774734 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:40:42.775086 9159 external_mini_cluster.cc:1428] Setting key c8ddfaae2db0141a9d1d6e4e8576214e
Aug 06 18:40:42 dist-test-slave-k5rr krb5kdc[24791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041642, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:40:42.780098 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.768862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:44851 (local address 127.8.241.254:34797)
0806 18:40:42.769060 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.769066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.769816 (+ 750us) server_negotiation.cc:407] Connection header received
0806 18:40:42.771139 (+ 1323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.771141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.771190 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.771263 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:42.772429 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.772966 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.774075 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.774281 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.774366 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.777074 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:42.777078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:42.777081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:42.777110 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:42.778308 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.778816 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.778818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.778819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.778891 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.779181 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.779183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.779184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.779356 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:42.779428 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.779875 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.779976 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:40:42.780953 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.769176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34797 (local address 127.8.241.195:44851)
0806 18:40:42.769657 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:42.769701 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:42.770944 (+ 1243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:42.771429 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:42.771436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:42.771817 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:40:42.772308 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.772322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.773090 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.773093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.773981 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.773991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.774393 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.774395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.774532 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.775118 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:40:42.775139 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:40:42.776888 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:40:42.778456 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:42.778462 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:42.778475 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:42.778731 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:42.778970 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:40:42.778972 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:40:42.778973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:40:42.779114 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:40:42.779429 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:40:42.779434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:40:42.779684 (+ 250us) client_negotiation.cc:769] Sending connection context
0806 18:40:42.779845 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 18:40:42.780115 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":116,"threads_started":1}
I20260806 18:40:42.782068 25591 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34797
I20260806 18:40:42.782307 25591 heartbeater.cc:464] Registering TS with master...
I20260806 18:40:42.782824 25591 heartbeater.cc:511] Master 127.8.241.254:34797 requested a full tablet report, sending...
I20260806 18:40:42.783859 24863 ts_manager.cc:194] Registered new tserver with Master: 2e105adccbe440feb65ae844fcbe44cf (127.8.241.195:39263)
I20260806 18:40:42.784480 24863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:44851
I20260806 18:40:42.788930 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:40:42.797640 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.790783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.8.241.254:34797)
0806 18:40:42.790908 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.790913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.791089 (+ 176us) server_negotiation.cc:407] Connection header received
0806 18:40:42.791225 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.791228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.791273 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.791332 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:40:42.792046 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.792562 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.793420 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.793641 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.793728 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.794952 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:40:42.794956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:40:42.794959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:40:42.794985 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:40:42.796011 (+ 1026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.796403 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.796406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.796408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.796501 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:40:42.796753 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:40:42.796757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:40:42.796758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:40:42.796985 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:40:42.797146 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.797363 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.797487 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:40:42.800274 24863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:40:42.803342 24863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:40:42.817621 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.812997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50244 (local address 127.8.241.195:39263)
0806 18:40:42.813688 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.813696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.813714 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:40:42.813780 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.813785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.813991 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.814133 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:42.814304 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.815095 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.816294 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.816904 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.817034 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.817189 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.817358 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.817472 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":120,"threads_started":1}
I20260806 18:40:42.817615 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.812821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:39263 (local address 127.0.0.1:50244)
0806 18:40:42.813368 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:42.813381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:42.813461 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:42.814116 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:42.814118 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:42.814137 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:42.814216 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.814222 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.815235 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.815239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.816168 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.816179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.817047 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.817051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.817163 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.817204 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:40:42.817319 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 18:40:42.817459 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260806 18:40:42.818284 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.812489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:32975 (local address 127.0.0.1:59226)
0806 18:40:42.813220 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:42.813233 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:42.813333 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:42.813811 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:42.813814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:42.813833 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:42.813932 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.813938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.815197 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.815200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.816681 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.816691 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.817907 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.817910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.817999 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.818018 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:40:42.818083 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:40:42.818153 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":148,"spinlock_wait_cycles":768,"thread_start_us":94,"threads_started":1}
I20260806 18:40:42.818362 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.812636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59226 (local address 127.8.241.193:32975)
0806 18:40:42.813074 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.813091 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.813410 (+ 319us) server_negotiation.cc:407] Connection header received
0806 18:40:42.813477 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.813481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.813672 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.813826 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:42.814027 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.815064 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.816821 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.817627 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.817725 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.817854 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.818137 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.818229 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":108,"threads_started":1}
I20260806 18:40:42.818440 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.811931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.8.241.194:45015)
0806 18:40:42.812413 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.812425 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.812450 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:40:42.812532 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.812538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.812764 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.812926 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:42.813575 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.814801 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.816814 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.817608 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.817716 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.817863 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.818187 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.818288 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":141,"threads_started":1}
I20260806 18:40:42.818805 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.811747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:45015 (local address 127.0.0.1:58244)
0806 18:40:42.812094 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:42.812149 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:42.812273 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:42.813106 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:42.813113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:42.813288 (+ 175us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:42.813455 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.813462 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.815072 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.815075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.816679 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.816691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.817925 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.817930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.818035 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.818059 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:40:42.818585 (+ 526us) client_negotiation.cc:240] Negotiation successful
0806 18:40:42.818673 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":104,"spinlock_wait_cycles":3968,"thread_start_us":84,"threads_started":1}
I20260806 18:40:42.821532 25276 tablet_service.cc:1467] Processing CreateTablet for tablet 7196f114b3684263abf4000459ca1722 (DEFAULT_TABLE table=test-table [id=340ff41c1ca04c398e766affcd80420d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:42.821631 25489 tablet_service.cc:1467] Processing CreateTablet for tablet 7196f114b3684263abf4000459ca1722 (DEFAULT_TABLE table=test-table [id=340ff41c1ca04c398e766affcd80420d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:42.821648 25057 tablet_service.cc:1467] Processing CreateTablet for tablet 7196f114b3684263abf4000459ca1722 (DEFAULT_TABLE table=test-table [id=340ff41c1ca04c398e766affcd80420d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:40:42.822624 25276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7196f114b3684263abf4000459ca1722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:42.822662 25057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7196f114b3684263abf4000459ca1722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:42.822674 25489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7196f114b3684263abf4000459ca1722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:40:42.828962 25614 tablet_bootstrap.cc:492] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: Bootstrap starting.
I20260806 18:40:42.830662 25619 tablet_bootstrap.cc:492] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a: Bootstrap starting.
I20260806 18:40:42.831637 25614 tablet_bootstrap.cc:654] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:42.832371 25614 log.cc:824] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:42.832413 25619 tablet_bootstrap.cc:654] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:42.833161 25616 tablet_bootstrap.cc:492] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: Bootstrap starting.
I20260806 18:40:42.833650 25619 log.cc:824] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:42.834937 25616 tablet_bootstrap.cc:654] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: Neither blocks nor log segments found. Creating new log.
I20260806 18:40:42.835359 25619 tablet_bootstrap.cc:492] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a: No bootstrap required, opened a new log
I20260806 18:40:42.835554 25619 ts_tablet_manager.cc:1397] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:40:42.835968 25616 log.cc:824] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: Log is configured to *not* fsync() on all Append() calls
I20260806 18:40:42.837510 25614 tablet_bootstrap.cc:492] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: No bootstrap required, opened a new log
I20260806 18:40:42.837713 25614 ts_tablet_manager.cc:1397] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260806 18:40:42.837896 25616 tablet_bootstrap.cc:492] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: No bootstrap required, opened a new log
I20260806 18:40:42.838080 25616 ts_tablet_manager.cc:1397] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:40:42.838660 25619 raft_consensus.cc:348] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } }
I20260806 18:40:42.838901 25619 raft_consensus.cc:374] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:42.838954 25619 raft_consensus.cc:729] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ff436ca3e04e2b81c611beefd6083a, State: Initialized, Role: FOLLOWER
I20260806 18:40:42.839367 25619 consensus_queue.cc:260] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } }
I20260806 18:40:42.840391 25619 ts_tablet_manager.cc:1428] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:40:42.840749 25172 heartbeater.cc:503] Master 127.8.241.254:34797 was elected leader, sending a full tablet report...
I20260806 18:40:42.841605 25614 raft_consensus.cc:348] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } }
I20260806 18:40:42.841873 25614 raft_consensus.cc:374] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:42.841929 25614 raft_consensus.cc:729] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d2d8677043e48ab8ed9ebefa79f2a7e, State: Initialized, Role: FOLLOWER
I20260806 18:40:42.841790 25616 raft_consensus.cc:348] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } }
I20260806 18:40:42.842005 25616 raft_consensus.cc:374] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:40:42.842056 25616 raft_consensus.cc:729] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e105adccbe440feb65ae844fcbe44cf, State: Initialized, Role: FOLLOWER
I20260806 18:40:42.842391 25614 consensus_queue.cc:260] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } }
I20260806 18:40:42.842391 25616 consensus_queue.cc:260] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } }
I20260806 18:40:42.843304 25591 heartbeater.cc:503] Master 127.8.241.254:34797 was elected leader, sending a full tablet report...
I20260806 18:40:42.843379 25616 ts_tablet_manager.cc:1428] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:40:42.843379 25614 ts_tablet_manager.cc:1428] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:40:42.843719 25384 heartbeater.cc:503] Master 127.8.241.254:34797 was elected leader, sending a full tablet report...
I20260806 18:40:42.931890 25624 raft_consensus.cc:482] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:40:42.932111 25624 raft_consensus.cc:504] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } }
I20260806 18:40:42.933445 25624 leader_election.cc:288] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75ff436ca3e04e2b81c611beefd6083a (127.8.241.193:32975), 2e105adccbe440feb65ae844fcbe44cf (127.8.241.195:39263)
I20260806 18:40:42.937455 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.933796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:42293 (local address 127.8.241.193:32975)
0806 18:40:42.933958 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.933964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.933981 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:40:42.934083 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.934087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.934146 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.934230 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:42.934523 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.935184 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.936384 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.936904 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.937031 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.937089 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.937269 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.937326 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:40:42.937476 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.933707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:32975 (local address 127.8.241.194:42293)
0806 18:40:42.933850 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:42.933868 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:42.933980 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:42.934247 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:42.934250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:42.934273 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:42.934396 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.934402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.935310 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.935314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.936258 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.936268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.937037 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.937041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.937148 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.937176 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:40:42.937255 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:40:42.937329 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 18:40:42.937857 25628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.933980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:39263 (local address 127.8.241.194:55749)
0806 18:40:42.934493 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:42.934506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:40:42.934606 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:40:42.934818 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:40:42.934821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:40:42.934839 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:40:42.934954 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.934960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.935775 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.935778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.936663 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:40:42.936674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.937440 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.937456 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:40:42.937557 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.937582 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:40:42.937656 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:40:42.937722 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":161,"threads_started":1}
I20260806 18:40:42.937856 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.934129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:55749 (local address 127.8.241.195:39263)
0806 18:40:42.934243 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.934247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.934571 (+ 324us) server_negotiation.cc:407] Connection header received
0806 18:40:42.934691 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.934694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.934734 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.934790 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:40:42.935047 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.935669 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.936786 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.937324 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.937431 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.937477 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.937681 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.937755 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:40:42.938252 25092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7196f114b3684263abf4000459ca1722" candidate_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ff436ca3e04e2b81c611beefd6083a" is_pre_election: true
I20260806 18:40:42.938542 25092 raft_consensus.cc:2454] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d2d8677043e48ab8ed9ebefa79f2a7e in term 0.
I20260806 18:40:42.938496 25521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7196f114b3684263abf4000459ca1722" candidate_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e105adccbe440feb65ae844fcbe44cf" is_pre_election: true
I20260806 18:40:42.938742 25521 raft_consensus.cc:2454] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d2d8677043e48ab8ed9ebefa79f2a7e in term 0.
I20260806 18:40:42.939038 25206 leader_election.cc:302] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ff436ca3e04e2b81c611beefd6083a, 8d2d8677043e48ab8ed9ebefa79f2a7e; no voters:
I20260806 18:40:42.939324 25624 raft_consensus.cc:2781] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:40:42.939409 25624 raft_consensus.cc:482] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:40:42.939440 25624 raft_consensus.cc:3037] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:42.940485 25624 raft_consensus.cc:504] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } }
I20260806 18:40:42.940832 25624 leader_election.cc:288] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [CANDIDATE]: Term 1 election: Requested vote from peers 75ff436ca3e04e2b81c611beefd6083a (127.8.241.193:32975), 2e105adccbe440feb65ae844fcbe44cf (127.8.241.195:39263)
I20260806 18:40:42.941244 25092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7196f114b3684263abf4000459ca1722" candidate_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75ff436ca3e04e2b81c611beefd6083a"
I20260806 18:40:42.941273 25521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7196f114b3684263abf4000459ca1722" candidate_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e105adccbe440feb65ae844fcbe44cf"
I20260806 18:40:42.941380 25521 raft_consensus.cc:3037] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:42.941375 25092 raft_consensus.cc:3037] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:40:42.942414 25521 raft_consensus.cc:2454] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d2d8677043e48ab8ed9ebefa79f2a7e in term 1.
I20260806 18:40:42.942415 25092 raft_consensus.cc:2454] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d2d8677043e48ab8ed9ebefa79f2a7e in term 1.
I20260806 18:40:42.942685 25206 leader_election.cc:302] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75ff436ca3e04e2b81c611beefd6083a, 8d2d8677043e48ab8ed9ebefa79f2a7e; no voters:
I20260806 18:40:42.942857 25624 raft_consensus.cc:2781] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:40:42.943089 25624 raft_consensus.cc:686] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 1 LEADER]: Becoming Leader. State: Replica: 8d2d8677043e48ab8ed9ebefa79f2a7e, State: Running, Role: LEADER
I20260806 18:40:42.943465 25624 consensus_queue.cc:237] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } }
I20260806 18:40:42.946676 24859 catalog_manager.cc:5495] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d2d8677043e48ab8ed9ebefa79f2a7e (127.8.241.194). New cstate: current_term: 1 leader_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d2d8677043e48ab8ed9ebefa79f2a7e" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 45015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 } health_report { overall_health: UNKNOWN } } }
I20260806 18:40:42.968080 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:42.964831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.8.241.194:45015)
0806 18:40:42.964949 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:40:42.964953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:40:42.964965 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:40:42.965122 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:40:42.965124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:40:42.965182 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:40:42.965266 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:40:42.965473 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.965953 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.966751 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:40:42.966978 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:40:42.967075 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:40:42.967329 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:40:42.967776 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:40:42.967868 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:40:42.967896 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 18:40:42.967948 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:40:42.974339 25092 raft_consensus.cc:1261] T 7196f114b3684263abf4000459ca1722 P 75ff436ca3e04e2b81c611beefd6083a [term 1 FOLLOWER]: Refusing update from remote peer 8d2d8677043e48ab8ed9ebefa79f2a7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:42.974339 25521 raft_consensus.cc:1261] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [term 1 FOLLOWER]: Refusing update from remote peer 8d2d8677043e48ab8ed9ebefa79f2a7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:40:42.975131 25633 consensus_queue.cc:1035] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e105adccbe440feb65ae844fcbe44cf" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:42.975327 25624 consensus_queue.cc:1035] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "75ff436ca3e04e2b81c611beefd6083a" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 32975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:40:42.984217 25640 mvcc.cc:205] Tried to move back new op lower bound from 7315626569614499840 to 7315626569499009024. Current Snapshot: MvccSnapshot[applied={T|T < 7315626569614499840}]
I20260806 18:40:42.993760 24859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:42.993927 24859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:40:42.996342 24859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5024e84c99dc4b4c82c53957e07215ea: Sending DeleteTablet for 3 replicas of tablet 7196f114b3684263abf4000459ca1722
I20260806 18:40:42.997102 25276 tablet_service.cc:1514] Processing DeleteTablet for tablet 7196f114b3684263abf4000459ca1722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:42 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:58244
I20260806 18:40:42.997457 25057 tablet_service.cc:1514] Processing DeleteTablet for tablet 7196f114b3684263abf4000459ca1722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:42 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:59226
I20260806 18:40:42.997635 25489 tablet_service.cc:1514] Processing DeleteTablet for tablet 7196f114b3684263abf4000459ca1722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:40:42 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:50244
I20260806 18:40:42.997728 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 24968
I20260806 18:40:42.998153 25653 tablet_replica.cc:330] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: stopping tablet replica
I20260806 18:40:42.998564 25653 raft_consensus.cc:2229] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:40:42.999063 25653 raft_consensus.cc:2258] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:43.007047 25657 tablet_replica.cc:330] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: stopping tablet replica
I20260806 18:40:43.007416 25657 raft_consensus.cc:2229] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:40:43.007771 25657 raft_consensus.cc:2258] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:40:43.009228 25653 ts_tablet_manager.cc:1905] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:43.009596 25657 ts_tablet_manager.cc:1905] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:40:43.012241 25653 ts_tablet_manager.cc:1918] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:43.012241 25657 ts_tablet_manager.cc:1918] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:40:43.012355 25653 log.cc:1196] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/7196f114b3684263abf4000459ca1722
I20260806 18:40:43.012496 25657 log.cc:1196] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/7196f114b3684263abf4000459ca1722
I20260806 18:40:43.012724 25653 ts_tablet_manager.cc:1939] T 7196f114b3684263abf4000459ca1722 P 8d2d8677043e48ab8ed9ebefa79f2a7e: Deleting consensus metadata
I20260806 18:40:43.012923 25657 ts_tablet_manager.cc:1939] T 7196f114b3684263abf4000459ca1722 P 2e105adccbe440feb65ae844fcbe44cf: Deleting consensus metadata
I20260806 18:40:43.014099 24841 catalog_manager.cc:4841] TS 2e105adccbe440feb65ae844fcbe44cf (127.8.241.195:39263): tablet 7196f114b3684263abf4000459ca1722 (table test-table [id=340ff41c1ca04c398e766affcd80420d]) successfully deleted
I20260806 18:40:43.014320 24841 catalog_manager.cc:4841] TS 8d2d8677043e48ab8ed9ebefa79f2a7e (127.8.241.194:45015): tablet 7196f114b3684263abf4000459ca1722 (table test-table [id=340ff41c1ca04c398e766affcd80420d]) successfully deleted
W20260806 18:40:43.017140 24841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:32975 (error 108)
I20260806 18:40:43.018040 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 25179
I20260806 18:40:43.018215 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:40:43.017907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:32975 (local address 127.0.0.1:59242)
0806 18:40:43.018059 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 18:40:43.018130 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:32975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 18:40:43.024986 24841 catalog_manager.cc:4584] TS 75ff436ca3e04e2b81c611beefd6083a (127.8.241.193:32975): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7196f114b3684263abf4000459ca1722: Network error: Client connection negotiation failed: client connection to 127.8.241.193:32975: connect: Connection refused (error 111)
I20260806 18:40:43.030647 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 25392
I20260806 18:40:43.044622 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 24810
2026-08-06T18:40:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2437 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:40:43 dist-test-slave-k5rr krb5kdc[25662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:40:43 dist-test-slave-k5rr krb5kdc[25662](info): set up 2 sockets
Aug 06 18:40:43 dist-test-slave-k5rr krb5kdc[25662](info): commencing operation
krb5kdc: starting...
W20260806 18:40:44.177574 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:40:44 dist-test-slave-k5rr krb5kdc[25662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:40:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:40:44Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/rangeradmin_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/rangeradmin_127.8.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/rangerlookup_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/rangerlookup_127.8.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/HTTP_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/HTTP_127.8.241.212@KRBTEST.COM.keytab.
I20260806 18:40:44.341434 9159 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5RraP2/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/postgres -l logfile start
2026-08-06 18:40:46.596 UTC [25707] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:40:46.596 UTC [25707] LOG: listening on IPv4 address "127.8.241.212", port 48261
2026-08-06 18:40:46.599 UTC [25707] LOG: listening on Unix socket "/tmp/.s.PGSQL.48261"
2026-08-06 18:40:46.618 UTC [25712] LOG: database system was shut down at 2026-08-06 18:40:45 UTC
2026-08-06 18:40:46.622 UTC [25707] LOG: database system is ready to accept connections
I20260806 18:40:47.669414 9159 mini_postgres.cc:96] Postgres bound to 48261
2026-08-06 18:40:47.676 UTC [25726] FATAL: database "slave" does not exist
127.8.241.212:48261 - accepting connections
I20260806 18:40:47.677475 9159 mini_ranger.cc:161] Starting Ranger...
I20260806 18:40:47.693179 9159 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:40:48.105645 9159 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:40:48.105743 9159 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-admin
2026-08-06 18:40:48,398 [I] DB FLAVOR :POSTGRES
2026-08-06 18:40:48,398 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:40:48,398 [I] Checking connection..
2026-08-06 18:40:48,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:40:48,659 [I] Checking connection passed.
2026-08-06 18:40:48,659 [I] --------- Verifying version history table ---------
2026-08-06 18:40:48,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:40:48,920 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:40:48,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:40:49,157 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:40:49,158 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:40:49,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:40:49.407 UTC [25825] WARNING: there is no transaction in progress
2026-08-06 18:40:49,413 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:40:49,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:40:49,652 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:40:49,652 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:40:49,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:40:49,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:40:50,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-k5rr.c.gcp-upstream.internal','N') ;"
2026-08-06 18:40:50,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:40:50,652 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:40:50,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:40:50,897 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:40:50,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:40:51,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:40:51,410 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:40:51,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:40:52.312 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.326 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.341 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.355 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.371 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.390 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.452 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.458 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.471 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.480 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.490 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.502 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.511 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.517 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.523 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:52.529 UTC [26010] WARNING: there is no transaction in progress
2026-08-06 18:40:53,104 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:40:53,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:40:53,368 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:40:53,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:40:53,632 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:40:53,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:40:53,876 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:40:53,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:40:54,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:40:54,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-k5rr.c.gcp-upstream.internal';"
2026-08-06 18:40:54,637 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:40:54,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:40:54,872 [I] DB_PATCHES have already been applied
I20260806 18:40:54.878036 9159 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-admin:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5RraP2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:40:54.878106 9159 mini_ranger.cc:193] Using host: 127.8.241.212
I20260806 18:40:54.882725 9159 mini_ranger.cc:227] Ranger admin URL: http://127.8.241.212:54869
Aug 06, 2026 6:40:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:40:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:40:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:40:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:40:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:40:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:40:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:40:56.091075 9159 mini_ranger.cc:160] Time spent starting Ranger: real 8.414s user 0.008s sys 0.003s
Aug 06, 2026 6:41:15 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:41:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:41:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:41:15 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:41:16 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:41:16.676383 9159 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/rangerkms_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/rangerkms_127.8.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.241.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.241.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.241.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/HTTP_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.241.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/HTTP_127.8.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:41:16 dist-test-slave-k5rr krb5kdc[25662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041676, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:41:16.772619 9159 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:41:16.785115 9159 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:41:17.605247 9159 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:41:17.605353 9159 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-kms
I20260806 18:41:17.605466 9159 mini_ranger_kms.cc:227] Using postgres at 127.8.241.212:48261
2026-08-06 18:41:18,675 [I] DB FLAVOR :POSTGRES
2026-08-06 18:41:18,675 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:41:18,675 [I] Checking connection
2026-08-06 18:41:18,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:41:19,056 [I] connection success
2026-08-06 18:41:19,056 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:41:19,057 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:41:19,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:41:19,321 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:41:19,322 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:41:19,322 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:41:19,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48261/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:41:19,585 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:41:20.042078 9159 mini_ranger_kms.cc:310] Created kms service
I20260806 18:41:20.264734 9159 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:41:20.308607 9159 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:41:20 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041676, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:41:20.728466 9159 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:41:20.728641 9159 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-kms:/tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:41:20.728663 9159 mini_ranger_kms.cc:240] Using host: 127.8.241.212
I20260806 18:41:20.733417 9159 mini_ranger_kms.cc:276] Ranger KMS PID: 26295
I20260806 18:41:20.733565 9159 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.241.212:37401
Aug 06, 2026 6:41:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:41:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:41:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:41:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:41:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:41:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 18:41:23.329536 9159 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.557s user 0.002s sys 0.959s
I20260806 18:41:23.329679 9159 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:41:23.329802 9159 mini_ranger_kms.cc:401] 127.8.241.212:37401/kms/v1/keys
Aug 06 18:41:23 dist-test-slave-k5rr krb5kdc[25662](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041683, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.241.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 18:41:24 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041683, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.241.212@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
Aug 06 18:41:24 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041683, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.241.212@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
Aug 06 18:41:27 dist-test-slave-k5rr krb5kdc[25662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:41:27.103909 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37191
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:37191
--ranger_config_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:37401/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:41:27.332683 26373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:41:27.332957 26373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:41:27.332998 26373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:41:27.337091 26373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:41:27.337146 26373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:41:27.337164 26373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:41:27.337183 26373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:41:27.337200 26373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:41:27.342808 26373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.241.212:37401/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:37191
--ranger_config_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:41:27.343837 26373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:41:27.344834 26373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:41:27.354817 26387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:41:27.355103 26373 server_base.cc:1034] running on GCE node
W20260806 18:41:27.354903 26381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:41:27.354949 26382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:41:27.355782 26373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:41:27.356895 26373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:41:27.358047 26373 hybrid_clock.cc:648] HybridClock initialized: now 1786041687358028 us; error 31 us; skew 500 ppm
Aug 06 18:41:27 dist-test-slave-k5rr krb5kdc[25662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041687, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:41:27.364593 26373 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:41:27.365640 26373 webserver.cc:460] Webserver started at http://127.8.241.254:34761/ using document root <none> and password file <none>
I20260806 18:41:27.366201 26373 fs_manager.cc:359] Metadata directory not provided
I20260806 18:41:27.366261 26373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:41:27.366482 26373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:41:27 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041687, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:41:27.430171 26373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "807b787abead4c57842a802061156971"
format_stamp: "Formatted at 2026-08-06 18:41:27 on dist-test-slave-k5rr"
server_key: "a9165ddb162b65fdd303f4b9cfb77e5f"
server_key_iv: "5a12ae2ed460cd88887887db92d6db62"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:27.430811 26373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "807b787abead4c57842a802061156971"
format_stamp: "Formatted at 2026-08-06 18:41:27 on dist-test-slave-k5rr"
server_key: "a9165ddb162b65fdd303f4b9cfb77e5f"
server_key_iv: "5a12ae2ed460cd88887887db92d6db62"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:27.435457 26373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 18:41:27.453280 26398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:27.454725 26373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:41:27.454861 26373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "807b787abead4c57842a802061156971"
format_stamp: "Formatted at 2026-08-06 18:41:27 on dist-test-slave-k5rr"
server_key: "a9165ddb162b65fdd303f4b9cfb77e5f"
server_key_iv: "5a12ae2ed460cd88887887db92d6db62"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:27.455001 26373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:41:27.471468 26373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:41:27.474745 26373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:41:27.475014 26373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:41:27.493745 26373 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:37191
I20260806 18:41:27.493919 26487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:37191 every 8 connection(s)
I20260806 18:41:27.494949 26373 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:41:27.495170 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 26373
I20260806 18:41:27.495316 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:41:27.495651 9159 external_mini_cluster.cc:1428] Setting key 833c77f13c014fd7f929de93e59d5475
I20260806 18:41:27.500396 26488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 18:41:27 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:41:27.508865 26488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971: Bootstrap starting.
I20260806 18:41:27.512334 26488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971: Neither blocks nor log segments found. Creating new log.
I20260806 18:41:27.514251 26488 log.cc:824] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971: Log is configured to *not* fsync() on all Append() calls
I20260806 18:41:27.514451 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:27.497552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.8.241.254:37191)
0806 18:41:27.498189 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:27.498243 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:27.498290 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:41:27.499531 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:27.499556 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:27.499986 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:27.500370 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:41:27.501269 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:27.503161 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:27.504442 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:27.505273 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:27.505441 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:27.508688 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:41:27.508695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:41:27.508712 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:41:27.508750 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:41:27.511008 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:27.511519 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:27.511525 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:27.511532 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:27.511673 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:27.511980 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:27.511983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:27.511985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:27.512548 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:41:27.512730 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:27.513137 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 18:41:27.513443 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":230,"threads_started":1}
I20260806 18:41:27.517622 26488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971: No bootstrap required, opened a new log
I20260806 18:41:27.522130 26488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b787abead4c57842a802061156971" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37191 } }
I20260806 18:41:27.522403 26488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:41:27.522480 26488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 807b787abead4c57842a802061156971, State: Initialized, Role: FOLLOWER
I20260806 18:41:27.522919 26488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b787abead4c57842a802061156971" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37191 } }
I20260806 18:41:27.523061 26488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:41:27.523128 26488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:41:27.523231 26488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:41:27.524425 26488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b787abead4c57842a802061156971" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37191 } }
I20260806 18:41:27.524880 26488 leader_election.cc:302] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 807b787abead4c57842a802061156971; no voters:
I20260806 18:41:27.525305 26488 leader_election.cc:288] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:41:27.525661 26494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:41:27.526083 26494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [term 1 LEADER]: Becoming Leader. State: Replica: 807b787abead4c57842a802061156971, State: Running, Role: LEADER
I20260806 18:41:27.526470 26494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b787abead4c57842a802061156971" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37191 } }
I20260806 18:41:27.526947 26488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:41:27.528491 26495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "807b787abead4c57842a802061156971" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b787abead4c57842a802061156971" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37191 } } }
I20260806 18:41:27.528626 26495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [sys.catalog]: This master's current role is: LEADER
I20260806 18:41:27.529191 26495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 807b787abead4c57842a802061156971. Latest consensus state: current_term: 1 leader_uuid: "807b787abead4c57842a802061156971" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b787abead4c57842a802061156971" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37191 } } }
I20260806 18:41:27.529284 26495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971 [sys.catalog]: This master's current role is: LEADER
I20260806 18:41:27.530432 26503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:41:27.530787 26488 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:41:27.534121 26503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:41:27.540484 26503 catalog_manager.cc:1347] Generated new cluster ID: c7dc7ec47d1c4e60aebce0d63966c511
I20260806 18:41:27.540560 26503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:41:27.555447 26503 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:41:27.557947 26503 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:41:27.568846 26503 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971: Generated new TSK 0
I20260806 18:41:27.569659 26503 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:41:28.598371 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:37401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37191
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:41:28.794533 26552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:41:28.794803 26552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:41:28.794867 26552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:41:28.800209 26552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:41:28.800284 26552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:41:28.800371 26552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:41:28.806358 26552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.241.212:37401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:41:28.807493 26552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:41:28.808562 26552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:41:28.822276 26557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:41:28.822389 26559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:41:28.822530 26561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:41:28.822700 26552 server_base.cc:1034] running on GCE node
I20260806 18:41:28.823156 26552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:41:28.823982 26552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:41:28.825148 26552 hybrid_clock.cc:648] HybridClock initialized: now 1786041688825113 us; error 52 us; skew 500 ppm
Aug 06 18:41:28 dist-test-slave-k5rr krb5kdc[25662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041688, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:41:28.828997 26552 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:41:28.830241 26552 webserver.cc:460] Webserver started at http://127.8.241.193:38851/ using document root <none> and password file <none>
I20260806 18:41:28.830920 26552 fs_manager.cc:359] Metadata directory not provided
I20260806 18:41:28.830988 26552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:41:28.831218 26552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:41:28 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041688, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:41:28.848019 26552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "71a0cca52f7643878f87e8550a616402"
format_stamp: "Formatted at 2026-08-06 18:41:28 on dist-test-slave-k5rr"
server_key: "bf2a1e24db76f5963a14c168601b7cea"
server_key_iv: "2f7a19f8f4daf0974991a3e234137617"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:28.848883 26552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "71a0cca52f7643878f87e8550a616402"
format_stamp: "Formatted at 2026-08-06 18:41:28 on dist-test-slave-k5rr"
server_key: "bf2a1e24db76f5963a14c168601b7cea"
server_key_iv: "2f7a19f8f4daf0974991a3e234137617"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:28.859628 26552 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.001s
I20260806 18:41:28.877902 26577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:28.879372 26552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:41:28.879511 26552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "71a0cca52f7643878f87e8550a616402"
format_stamp: "Formatted at 2026-08-06 18:41:28 on dist-test-slave-k5rr"
server_key: "bf2a1e24db76f5963a14c168601b7cea"
server_key_iv: "2f7a19f8f4daf0974991a3e234137617"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:28.879653 26552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:41:28.900746 26552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:41:28.903784 26552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:41:28.903992 26552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:41:28.904668 26552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:41:28.905743 26552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:41:28.905797 26552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:28.905850 26552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:41:28.905905 26552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:28.932770 26552 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:44849
I20260806 18:41:28.934330 26552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:41:28.936569 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 26552
I20260806 18:41:28.936721 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:41:28.937059 9159 external_mini_cluster.cc:1428] Setting key 9500340ef15cdfbc103eeb424a3156c0
I20260806 18:41:28.945434 26752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:44849 every 8 connection(s)
Aug 06 18:41:28 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041688, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:41:28.973362 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:28.946772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:38729 (local address 127.8.241.254:37191)
0806 18:41:28.951148 (+ 4376us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:28.951154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:28.951169 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:41:28.951218 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:28.951221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:28.951394 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:28.951539 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:41:28.953709 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:28.954503 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:28.959145 (+ 4642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:28.959520 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:28.959608 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:28.968097 (+ 8489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:41:28.968102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:41:28.968105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:41:28.968141 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:41:28.970426 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:28.971207 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:28.971211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:28.971213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:28.971317 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:28.971821 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:28.971825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:28.971827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:28.972098 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:41:28.972189 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:28.972986 (+ 797us) server_negotiation.cc:299] Negotiation successful
0806 18:41:28.973160 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4285,"thread_start_us":133,"threads_started":1}
I20260806 18:41:28.975569 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:28.946334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37191 (local address 127.8.241.193:38729)
0806 18:41:28.949085 (+ 2751us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:28.949147 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:41:28.950663 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:41:28.951962 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:41:28.951974 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:41:28.952603 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:41:28.953522 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:28.953544 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:28.954657 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:28.954661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:28.956279 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:28.956292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:28.959876 (+ 3584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:28.959881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:28.960265 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:28.961208 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:41:28.961231 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:41:28.967814 (+ 6583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:41:28.970607 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:41:28.970614 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:41:28.970637 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:41:28.971074 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:41:28.971441 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:41:28.971444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:41:28.971446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:41:28.971680 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:41:28.972267 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:41:28.972275 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:41:28.972662 (+ 387us) client_negotiation.cc:769] Sending connection context
0806 18:41:28.973917 (+ 1255us) client_negotiation.cc:240] Negotiation successful
0806 18:41:28.974318 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2462,"thread_start_us":1454,"threads_started":1}
I20260806 18:41:28.977779 26753 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37191
I20260806 18:41:28.978093 26753 heartbeater.cc:464] Registering TS with master...
I20260806 18:41:28.978758 26753 heartbeater.cc:511] Master 127.8.241.254:37191 requested a full tablet report, sending...
I20260806 18:41:28.980902 26422 ts_manager.cc:194] Registered new tserver with Master: 71a0cca52f7643878f87e8550a616402 (127.8.241.193:44849)
I20260806 18:41:28.982559 26422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:38729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:41:29.027545 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:37401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37191
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:41:29 dist-test-slave-k5rr krb5kdc[25662](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041689, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:41:29.267217 26767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:41:29.267711 26767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:41:29.267839 26767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:41:29.274088 26767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:41:29.274279 26767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:41:29.274426 26767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:41:29.282737 26767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.241.212:37401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:41:29.284407 26767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:41:29.285897 26767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:41:29.305832 26780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:41:29.306118 26778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:41:29.307440 26767 server_base.cc:1034] running on GCE node
W20260806 18:41:29.307730 26777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:41:29.308347 26767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:41:29.309237 26767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:41:29.310395 26767 hybrid_clock.cc:648] HybridClock initialized: now 1786041689310330 us; error 79 us; skew 500 ppm
Aug 06 18:41:29 dist-test-slave-k5rr krb5kdc[25662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041689, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:41:29.314802 26767 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:41:29.315929 26767 webserver.cc:460] Webserver started at http://127.8.241.194:40001/ using document root <none> and password file <none>
I20260806 18:41:29.316510 26767 fs_manager.cc:359] Metadata directory not provided
I20260806 18:41:29.316579 26767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:41:29.316804 26767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:41:29 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041689, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:41:29.337096 26767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "96053e7213504addb800a6ee816bab89"
format_stamp: "Formatted at 2026-08-06 18:41:29 on dist-test-slave-k5rr"
server_key: "d9a056920bc9e10df1a8f6b8e282f446"
server_key_iv: "4f02438760776b7ba2c9beba2069f0b4"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:29.339239 26767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "96053e7213504addb800a6ee816bab89"
format_stamp: "Formatted at 2026-08-06 18:41:29 on dist-test-slave-k5rr"
server_key: "d9a056920bc9e10df1a8f6b8e282f446"
server_key_iv: "4f02438760776b7ba2c9beba2069f0b4"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:29.344244 26767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 18:41:29.372426 26791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:29.374440 26767 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 18:41:29.374605 26767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "96053e7213504addb800a6ee816bab89"
format_stamp: "Formatted at 2026-08-06 18:41:29 on dist-test-slave-k5rr"
server_key: "d9a056920bc9e10df1a8f6b8e282f446"
server_key_iv: "4f02438760776b7ba2c9beba2069f0b4"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:29.374787 26767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:41:29.412142 26767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:41:29.416496 26767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:41:29.416815 26767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:41:29.417809 26767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:41:29.419200 26767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:41:29.419314 26767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:29.419447 26767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:41:29.419523 26767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:29.448278 26767 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:38203
I20260806 18:41:29.449993 26767 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:41:29.453473 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 26767
I20260806 18:41:29.453629 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:41:29.453996 9159 external_mini_cluster.cc:1428] Setting key f38a7cb821e3cb27db82dc92c8a8de6c
I20260806 18:41:29.452505 26953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:38203 every 8 connection(s)
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Aug 06 18:41:29 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041689, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:41:29.494020 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.461800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:59771 (local address 127.8.241.254:37191)
0806 18:41:29.461951 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:29.461957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:29.466707 (+ 4750us) server_negotiation.cc:407] Connection header received
0806 18:41:29.476259 (+ 9552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:29.476264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:29.476351 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:29.476422 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:41:29.478987 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.479846 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.481780 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.482114 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.482204 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.487335 (+ 5131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:41:29.487339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:41:29.487342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:41:29.487373 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:41:29.489186 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:29.489879 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:29.489883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:29.489884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:29.489986 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:29.490447 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:29.490451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:29.490452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:29.490702 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:41:29.490790 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:29.493652 (+ 2862us) server_negotiation.cc:299] Negotiation successful
0806 18:41:29.493809 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:41:29.495791 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.460344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37191 (local address 127.8.241.194:59771)
0806 18:41:29.466520 (+ 6176us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:29.466586 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 18:41:29.468148 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:41:29.477062 (+ 8914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:41:29.477076 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:41:29.477745 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:41:29.478753 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.478782 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.480031 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.480035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.481621 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.481635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.482451 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.482456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.482747 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.484562 (+ 1815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:41:29.484585 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:41:29.487100 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:41:29.489338 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:41:29.489344 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:41:29.489363 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:41:29.489748 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:41:29.490107 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:41:29.490110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:41:29.490112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:41:29.490321 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:41:29.490866 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:41:29.490873 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:41:29.491228 (+ 355us) client_negotiation.cc:769] Sending connection context
0806 18:41:29.494332 (+ 3104us) client_negotiation.cc:240] Negotiation successful
0806 18:41:29.494740 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5900,"thread_start_us":333,"threads_started":1}
I20260806 18:41:29.497133 26954 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37191
I20260806 18:41:29.497407 26954 heartbeater.cc:464] Registering TS with master...
I20260806 18:41:29.497990 26954 heartbeater.cc:511] Master 127.8.241.254:37191 requested a full tablet report, sending...
I20260806 18:41:29.499082 26422 ts_manager.cc:194] Registered new tserver with Master: 96053e7213504addb800a6ee816bab89 (127.8.241.194:38203)
I20260806 18:41:29.499830 26422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:59771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:41:29.520642 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:37401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37191
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:41:29.724990 26961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:41:29.725324 26961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:41:29.725389 26961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:41:29.730907 26961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:41:29.731060 26961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:41:29.731181 26961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:41:29.737987 26961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.241.212:37401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:41:29.739537 26961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:41:29.741158 26961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:41:29.754837 26971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:41:29.754769 26969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:41:29.755115 26968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:41:29.756283 26961 server_base.cc:1034] running on GCE node
I20260806 18:41:29.756701 26961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:41:29.757484 26961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:41:29.758670 26961 hybrid_clock.cc:648] HybridClock initialized: now 1786041689758640 us; error 43 us; skew 500 ppm
Aug 06 18:41:29 dist-test-slave-k5rr krb5kdc[25662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041689, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:41:29.762449 26961 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:41:29.763440 26961 webserver.cc:460] Webserver started at http://127.8.241.195:33939/ using document root <none> and password file <none>
I20260806 18:41:29.764005 26961 fs_manager.cc:359] Metadata directory not provided
I20260806 18:41:29.764071 26961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:41:29.764308 26961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:41:29 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041689, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:41:29.785851 26961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "5000779a998641aab276af27fcf42f8f"
format_stamp: "Formatted at 2026-08-06 18:41:29 on dist-test-slave-k5rr"
server_key: "2f644a7f931c1633db5eb2ed3c4ba4b9"
server_key_iv: "b8bcb0bfbd3150a7dfff2f180867c1ac"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:29.786696 26961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "5000779a998641aab276af27fcf42f8f"
format_stamp: "Formatted at 2026-08-06 18:41:29 on dist-test-slave-k5rr"
server_key: "2f644a7f931c1633db5eb2ed3c4ba4b9"
server_key_iv: "b8bcb0bfbd3150a7dfff2f180867c1ac"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:29.804266 26961 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.006s sys 0.000s
Aug 06 18:41:29 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041689, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:41:29.839371 26980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:29.841707 26961 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.002s
I20260806 18:41:29.841869 26961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "5000779a998641aab276af27fcf42f8f"
format_stamp: "Formatted at 2026-08-06 18:41:29 on dist-test-slave-k5rr"
server_key: "2f644a7f931c1633db5eb2ed3c4ba4b9"
server_key_iv: "b8bcb0bfbd3150a7dfff2f180867c1ac"
server_key_version: "kuduclusterkey@0"
I20260806 18:41:29.842008 26961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:41:29.868079 26961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:41:29.873764 26961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:41:29.874109 26961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:41:29.875118 26961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:41:29.876507 26961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:41:29.876569 26961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:29.876652 26961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:41:29.876682 26961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:41:29.906435 26961 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:45115
I20260806 18:41:29.907548 27164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:45115 every 8 connection(s)
I20260806 18:41:29.907881 26961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:41:29.912364 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 26961
I20260806 18:41:29.912492 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:41:29.912789 9159 external_mini_cluster.cc:1428] Setting key 054e6055b9363c19f17498c716618e93
Aug 06 18:41:29 dist-test-slave-k5rr krb5kdc[25662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041689, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:41:29.929880 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.912403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:40191 (local address 127.8.241.254:37191)
0806 18:41:29.912576 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:29.912581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:29.917670 (+ 5089us) server_negotiation.cc:407] Connection header received
0806 18:41:29.919061 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:29.919064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:29.919121 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:29.919193 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:41:29.920717 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.921307 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.922581 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.922800 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.922891 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.925950 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:41:29.925955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:41:29.925957 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:41:29.925988 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:41:29.927640 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:29.928186 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:29.928189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:29.928191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:29.928273 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:29.928629 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:29.928633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:29.928634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:29.928825 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:41:29.928914 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:29.929561 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 18:41:29.929719 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:41:29.930817 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.911250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37191 (local address 127.8.241.195:40191)
0806 18:41:29.917519 (+ 6269us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:29.917574 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:41:29.918834 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:41:29.919394 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:41:29.919402 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:41:29.919874 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:41:29.920568 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.920585 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.921437 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.921440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.922460 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.922471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.922958 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.922963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.923187 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.923822 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:41:29.923839 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:41:29.925766 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:41:29.927741 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:41:29.927767 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:41:29.927786 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:41:29.928093 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:41:29.928358 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:41:29.928361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:41:29.928363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:41:29.928525 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:41:29.928929 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:41:29.928936 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:41:29.929260 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 18:41:29.929443 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 18:41:29.929781 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5990,"thread_start_us":257,"threads_started":1}
I20260806 18:41:29.931996 27165 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37191
I20260806 18:41:29.932253 27165 heartbeater.cc:464] Registering TS with master...
I20260806 18:41:29.932785 27165 heartbeater.cc:511] Master 127.8.241.254:37191 requested a full tablet report, sending...
I20260806 18:41:29.933871 26422 ts_manager.cc:194] Registered new tserver with Master: 5000779a998641aab276af27fcf42f8f (127.8.241.195:45115)
I20260806 18:41:29.934438 26422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:40191
I20260806 18:41:29.938340 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:41:29.949358 26759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.940331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.8.241.254:37191)
0806 18:41:29.940444 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:29.940449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:29.940970 (+ 521us) server_negotiation.cc:407] Connection header received
0806 18:41:29.941171 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:29.941174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:29.941235 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:29.941306 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:41:29.942223 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.942753 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.943588 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.943837 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.943939 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.945747 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:41:29.945752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:41:29.945755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:41:29.945785 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:41:29.947209 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:29.947723 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:29.947728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:29.947729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:29.947817 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:41:29.948105 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:41:29.948109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:41:29.948110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:41:29.948342 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:41:29.948481 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:29.949049 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 18:41:29.949179 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:41:29.952731 26422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:41:29.956437 26422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:41:29.976379 27185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.968021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:38203 (local address 127.0.0.1:59058)
0806 18:41:29.969227 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:29.969287 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:41:29.969450 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:41:29.970445 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:41:29.970452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:41:29.970481 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:41:29.970649 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.970659 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.972527 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.972531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.974010 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.974024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.975722 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.975727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.975868 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.975923 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 18:41:29.976054 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 18:41:29.976201 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"thread_start_us":240,"threads_started":1}
I20260806 18:41:29.977494 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.968183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59058 (local address 127.8.241.194:38203)
0806 18:41:29.968747 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:29.968808 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:29.969664 (+ 856us) server_negotiation.cc:407] Connection header received
0806 18:41:29.969773 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:29.969779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:29.970005 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:29.970233 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:41:29.970878 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.972354 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.974301 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.975334 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.975455 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.975633 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:29.977156 (+ 1523us) server_negotiation.cc:299] Negotiation successful
0806 18:41:29.977286 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":176,"threads_started":1}
I20260806 18:41:29.980835 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.969333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44849 (local address 127.0.0.1:60892)
0806 18:41:29.972540 (+ 3207us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:29.972555 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:41:29.972672 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:41:29.973998 (+ 1326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:41:29.974001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:41:29.974029 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:41:29.974237 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.974245 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.976602 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.976605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.977966 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.977979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.980325 (+ 2346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.980329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.980504 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.980532 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:41:29.980605 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:41:29.980680 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3139,"thread_start_us":168,"threads_started":1}
I20260806 18:41:29.981431 27189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.969918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.8.241.193:44849)
0806 18:41:29.973282 (+ 3364us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:29.973289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:29.973313 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:41:29.973389 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:29.973395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:29.973603 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:29.973740 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:41:29.974382 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.976461 (+ 2079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.978110 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.979982 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.980089 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.980255 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:29.981150 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 18:41:29.981255 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3244,"thread_start_us":84,"threads_started":1}
I20260806 18:41:29.981870 26861 tablet_service.cc:1467] Processing CreateTablet for tablet 9cedea8f366447d68a776e535aa97778 (DEFAULT_TABLE table=test-table [id=93630ee97a884fc79d82185992408a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:41:29.983179 26861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cedea8f366447d68a776e535aa97778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:41:29.984448 26654 tablet_service.cc:1467] Processing CreateTablet for tablet 9cedea8f366447d68a776e535aa97778 (DEFAULT_TABLE table=test-table [id=93630ee97a884fc79d82185992408a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:41:29.985898 26753 heartbeater.cc:503] Master 127.8.241.254:37191 was elected leader, sending a full tablet report...
I20260806 18:41:29.986918 26654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cedea8f366447d68a776e535aa97778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:41:29.993096 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.970635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:45115 (local address 127.0.0.1:41666)
0806 18:41:29.972890 (+ 2255us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:29.972902 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:41:29.973033 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:41:29.981703 (+ 8670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:41:29.981706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:41:29.981729 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:41:29.981826 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.981833 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.989527 (+ 7694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.989531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.991155 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:29.991170 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.992625 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.992629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:29.992731 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.992765 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 18:41:29.992827 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:41:29.992909 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2196,"thread_start_us":166,"threads_started":1}
I20260806 18:41:29.994148 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:29.971506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.8.241.195:45115)
0806 18:41:29.980949 (+ 9443us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:29.981048 (+ 99us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:29.981081 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:41:29.981171 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:29.981176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:29.981485 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:29.981990 (+ 505us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:41:29.982133 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.986464 (+ 4331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.991430 (+ 4966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:29.992476 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:29.993588 (+ 1112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:29.993763 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:29.993836 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 18:41:29.993940 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9308,"thread_start_us":220,"threads_started":1}
I20260806 18:41:29.999136 27194 tablet_bootstrap.cc:492] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402: Bootstrap starting.
I20260806 18:41:29.998802 27062 tablet_service.cc:1467] Processing CreateTablet for tablet 9cedea8f366447d68a776e535aa97778 (DEFAULT_TABLE table=test-table [id=93630ee97a884fc79d82185992408a58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:41:30.000195 27062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cedea8f366447d68a776e535aa97778. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:41:30.000605 27192 tablet_bootstrap.cc:492] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89: Bootstrap starting.
I20260806 18:41:30.002933 27194 tablet_bootstrap.cc:654] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402: Neither blocks nor log segments found. Creating new log.
I20260806 18:41:30.003767 27192 tablet_bootstrap.cc:654] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89: Neither blocks nor log segments found. Creating new log.
I20260806 18:41:30.005914 27194 log.cc:824] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402: Log is configured to *not* fsync() on all Append() calls
I20260806 18:41:30.005914 27192 log.cc:824] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89: Log is configured to *not* fsync() on all Append() calls
I20260806 18:41:30.008320 27192 tablet_bootstrap.cc:492] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89: No bootstrap required, opened a new log
I20260806 18:41:30.008633 27192 ts_tablet_manager.cc:1397] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260806 18:41:30.009287 27194 tablet_bootstrap.cc:492] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402: No bootstrap required, opened a new log
I20260806 18:41:30.009517 27194 ts_tablet_manager.cc:1397] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20260806 18:41:30.012811 27198 tablet_bootstrap.cc:492] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: Bootstrap starting.
I20260806 18:41:30.015659 27194 raft_consensus.cc:348] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } }
I20260806 18:41:30.016017 27194 raft_consensus.cc:374] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:41:30.016111 27194 raft_consensus.cc:729] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71a0cca52f7643878f87e8550a616402, State: Initialized, Role: FOLLOWER
I20260806 18:41:30.016436 27192 raft_consensus.cc:348] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } }
I20260806 18:41:30.016731 27192 raft_consensus.cc:374] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:41:30.016683 27194 consensus_queue.cc:260] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } }
I20260806 18:41:30.016830 27192 raft_consensus.cc:729] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96053e7213504addb800a6ee816bab89, State: Initialized, Role: FOLLOWER
I20260806 18:41:30.017364 27198 tablet_bootstrap.cc:654] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: Neither blocks nor log segments found. Creating new log.
I20260806 18:41:30.017432 27192 consensus_queue.cc:260] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } }
I20260806 18:41:30.018038 27194 ts_tablet_manager.cc:1428] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402: Time spent starting tablet: real 0.008s user 0.003s sys 0.005s
I20260806 18:41:30.018725 27192 ts_tablet_manager.cc:1428] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89: Time spent starting tablet: real 0.010s user 0.002s sys 0.006s
I20260806 18:41:30.019007 27198 log.cc:824] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:41:30.019121 26954 heartbeater.cc:503] Master 127.8.241.254:37191 was elected leader, sending a full tablet report...
I20260806 18:41:30.022136 27198 tablet_bootstrap.cc:492] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: No bootstrap required, opened a new log
I20260806 18:41:30.022368 27198 ts_tablet_manager.cc:1397] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 18:41:30.026942 27200 raft_consensus.cc:482] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:41:30.027256 27200 raft_consensus.cc:504] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } }
I20260806 18:41:30.028640 27198 raft_consensus.cc:348] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } }
I20260806 18:41:30.029110 27198 raft_consensus.cc:374] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:41:30.029227 27198 raft_consensus.cc:729] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5000779a998641aab276af27fcf42f8f, State: Initialized, Role: FOLLOWER
I20260806 18:41:30.029837 27198 consensus_queue.cc:260] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } }
I20260806 18:41:30.030251 27200 leader_election.cc:288] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96053e7213504addb800a6ee816bab89 (127.8.241.194:38203), 5000779a998641aab276af27fcf42f8f (127.8.241.195:45115)
I20260806 18:41:30.031095 27165 heartbeater.cc:503] Master 127.8.241.254:37191 was elected leader, sending a full tablet report...
I20260806 18:41:30.031445 27198 ts_tablet_manager.cc:1428] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260806 18:41:30.036870 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:30.029133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:49655 (local address 127.8.241.194:38203)
0806 18:41:30.029254 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:30.029259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:30.032005 (+ 2746us) server_negotiation.cc:407] Connection header received
0806 18:41:30.032060 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:30.032063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:30.032113 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:30.032178 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:41:30.032945 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.033696 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.035426 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.036048 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.036138 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:30.036191 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:30.036640 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 18:41:30.036736 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:41:30.037686 27205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:30.029112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:38203 (local address 127.8.241.193:49655)
0806 18:41:30.031753 (+ 2641us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:30.031769 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:41:30.031913 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:41:30.032302 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:41:30.032305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:41:30.032324 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:41:30.032794 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:30.032805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.033845 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.033848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:30.035276 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:30.035287 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.036359 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.036362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:30.036500 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:30.036528 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:41:30.037269 (+ 741us) client_negotiation.cc:240] Negotiation successful
0806 18:41:30.037352 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2555,"thread_start_us":175,"threads_started":1}
I20260806 18:41:30.037842 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:30.030376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:44719 (local address 127.8.241.195:45115)
0806 18:41:30.030512 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:30.030516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:30.032927 (+ 2411us) server_negotiation.cc:407] Connection header received
0806 18:41:30.033127 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:30.033130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:30.033175 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:30.033231 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:41:30.033847 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.034590 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.036513 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.037090 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.037174 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:30.037232 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:30.037643 (+ 411us) server_negotiation.cc:299] Negotiation successful
0806 18:41:30.037728 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:41:30.038235 27204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:30.029730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:45115 (local address 127.8.241.193:44719)
0806 18:41:30.032818 (+ 3088us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:30.032831 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:41:30.032992 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:41:30.033359 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:41:30.033362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:41:30.033379 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:41:30.033722 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:30.033730 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.034857 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.034861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:30.036375 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:41:30.036383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.037321 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.037324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:41:30.037491 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:30.037517 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:41:30.037962 (+ 445us) client_negotiation.cc:240] Negotiation successful
0806 18:41:30.038039 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3029,"thread_start_us":152,"threads_started":1}
I20260806 18:41:30.038828 26891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cedea8f366447d68a776e535aa97778" candidate_uuid: "71a0cca52f7643878f87e8550a616402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96053e7213504addb800a6ee816bab89" is_pre_election: true
I20260806 18:41:30.039143 27082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cedea8f366447d68a776e535aa97778" candidate_uuid: "71a0cca52f7643878f87e8550a616402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5000779a998641aab276af27fcf42f8f" is_pre_election: true
I20260806 18:41:30.039208 26891 raft_consensus.cc:2454] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71a0cca52f7643878f87e8550a616402 in term 0.
I20260806 18:41:30.039427 27082 raft_consensus.cc:2454] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71a0cca52f7643878f87e8550a616402 in term 0.
I20260806 18:41:30.040077 26578 leader_election.cc:302] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71a0cca52f7643878f87e8550a616402, 96053e7213504addb800a6ee816bab89; no voters:
I20260806 18:41:30.040546 27200 raft_consensus.cc:2781] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:41:30.040618 27200 raft_consensus.cc:482] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:41:30.040659 27200 raft_consensus.cc:3037] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:41:30.042790 27200 raft_consensus.cc:504] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } }
I20260806 18:41:30.043219 27200 leader_election.cc:288] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [CANDIDATE]: Term 1 election: Requested vote from peers 96053e7213504addb800a6ee816bab89 (127.8.241.194:38203), 5000779a998641aab276af27fcf42f8f (127.8.241.195:45115)
I20260806 18:41:30.043737 26891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cedea8f366447d68a776e535aa97778" candidate_uuid: "71a0cca52f7643878f87e8550a616402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96053e7213504addb800a6ee816bab89"
I20260806 18:41:30.043779 27082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cedea8f366447d68a776e535aa97778" candidate_uuid: "71a0cca52f7643878f87e8550a616402" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5000779a998641aab276af27fcf42f8f"
I20260806 18:41:30.043882 27082 raft_consensus.cc:3037] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:41:30.043882 26891 raft_consensus.cc:3037] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:41:30.045925 26891 raft_consensus.cc:2454] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71a0cca52f7643878f87e8550a616402 in term 1.
I20260806 18:41:30.045925 27082 raft_consensus.cc:2454] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71a0cca52f7643878f87e8550a616402 in term 1.
I20260806 18:41:30.046420 26581 leader_election.cc:302] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5000779a998641aab276af27fcf42f8f, 71a0cca52f7643878f87e8550a616402, 96053e7213504addb800a6ee816bab89; no voters:
I20260806 18:41:30.046679 27200 raft_consensus.cc:2781] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:41:30.046942 27200 raft_consensus.cc:686] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [term 1 LEADER]: Becoming Leader. State: Replica: 71a0cca52f7643878f87e8550a616402, State: Running, Role: LEADER
I20260806 18:41:30.047339 27200 consensus_queue.cc:237] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } }
I20260806 18:41:30.051556 26422 catalog_manager.cc:5495] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71a0cca52f7643878f87e8550a616402 (127.8.241.193). New cstate: current_term: 1 leader_uuid: "71a0cca52f7643878f87e8550a616402" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71a0cca52f7643878f87e8550a616402" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 } health_report { overall_health: UNKNOWN } } }
I20260806 18:41:30.076256 27189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:30.070912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.8.241.193:44849)
0806 18:41:30.071060 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:41:30.071066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:41:30.071090 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:41:30.071330 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:41:30.071334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:41:30.071412 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:41:30.071555 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:41:30.072013 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.072826 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.074153 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:41:30.074507 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:41:30.074619 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:41:30.075016 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:41:30.075650 (+ 634us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:41:30.075788 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:41:30.075897 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 18:41:30.075972 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:41:30.088100 26891 raft_consensus.cc:1261] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [term 1 FOLLOWER]: Refusing update from remote peer 71a0cca52f7643878f87e8550a616402: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:41:30.088100 27082 raft_consensus.cc:1261] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [term 1 FOLLOWER]: Refusing update from remote peer 71a0cca52f7643878f87e8550a616402: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:41:30.088914 27200 consensus_queue.cc:1035] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96053e7213504addb800a6ee816bab89" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:41:30.089493 27208 consensus_queue.cc:1035] T 9cedea8f366447d68a776e535aa97778 P 71a0cca52f7643878f87e8550a616402 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5000779a998641aab276af27fcf42f8f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:41:30.105693 27215 mvcc.cc:205] Tried to move back new op lower bound from 7315626762589630464 to 7315626762437873664. Current Snapshot: MvccSnapshot[applied={T|T < 7315626762589630464}]
I20260806 18:41:30.111215 27220 mvcc.cc:205] Tried to move back new op lower bound from 7315626762589630464 to 7315626762437873664. Current Snapshot: MvccSnapshot[applied={T|T < 7315626762589630464 or (T in {7315626762589630464})}]
I20260806 18:41:30.127810 26418 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:41:30.128005 26418 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:41:30.130956 26418 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 807b787abead4c57842a802061156971: Sending DeleteTablet for 3 replicas of tablet 9cedea8f366447d68a776e535aa97778
I20260806 18:41:30.131776 26861 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cedea8f366447d68a776e535aa97778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:41:30 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:59058
I20260806 18:41:30.132120 26654 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cedea8f366447d68a776e535aa97778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:41:30 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:60892
I20260806 18:41:30.133209 27062 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cedea8f366447d68a776e535aa97778 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:41:30 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:41666
I20260806 18:41:30.134371 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 26552
I20260806 18:41:30.141436 27235 tablet_replica.cc:330] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: stopping tablet replica
I20260806 18:41:30.141954 27235 raft_consensus.cc:2229] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:41:30.142380 27235 raft_consensus.cc:2258] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:41:30.145756 27235 ts_tablet_manager.cc:1905] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:41:30.149487 27235 ts_tablet_manager.cc:1918] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:41:30.149597 27235 log.cc:1196] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/9cedea8f366447d68a776e535aa97778
I20260806 18:41:30.150004 27235 ts_tablet_manager.cc:1939] T 9cedea8f366447d68a776e535aa97778 P 5000779a998641aab276af27fcf42f8f: Deleting consensus metadata
I20260806 18:41:30.151444 26406 catalog_manager.cc:4841] TS 5000779a998641aab276af27fcf42f8f (127.8.241.195:45115): tablet 9cedea8f366447d68a776e535aa97778 (table test-table [id=93630ee97a884fc79d82185992408a58]) successfully deleted
I20260806 18:41:30.153348 27233 tablet_replica.cc:330] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89: stopping tablet replica
I20260806 18:41:30.153667 27233 raft_consensus.cc:2229] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:41:30.153995 27233 raft_consensus.cc:2258] T 9cedea8f366447d68a776e535aa97778 P 96053e7213504addb800a6ee816bab89 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 18:41:30.154435 26400 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:44849 (error 108)
W20260806 18:41:30.155633 27187 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:41:30.155189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44849 (local address 127.0.0.1:60914)
0806 18:41:30.155471 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:30.155539 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44849: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":203}
I20260806 18:41:30.155894 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 26767
W20260806 18:41:30.155900 26400 catalog_manager.cc:4584] TS 71a0cca52f7643878f87e8550a616402 (127.8.241.193:44849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9cedea8f366447d68a776e535aa97778: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44849: connect: Connection reset by peer (error 104)
I20260806 18:41:30.172722 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 26961
I20260806 18:41:30.172822 27187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:41:30.172412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:38203 (local address 127.0.0.1:59060)
0806 18:41:30.172652 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 18:41:30.172727 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.194:38203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260806 18:41:30.192649 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 26373
I20260806 18:41:30.209511 9159 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:41:30.553784 9159 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:41:30.554003 9159 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:41:30.898350 9159 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:41:30.898 UTC [25707] LOG: received smart shutdown request
2026-08-06 18:41:30.901 UTC [25707] LOG: background worker "logical replication launcher" (PID 25724) exited with exit code 1
2026-08-06 18:41:30.901 UTC [25713] LOG: shutting down
2026-08-06 18:41:31.042 UTC [25707] LOG: database system is shut down
2026-08-06T18:41:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48092 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:41:31 dist-test-slave-k5rr krb5kdc[27252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:41:31 dist-test-slave-k5rr krb5kdc[27252](info): set up 2 sockets
Aug 06 18:41:31 dist-test-slave-k5rr krb5kdc[27252](info): commencing operation
krb5kdc: starting...
W20260806 18:41:32.298169 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:41:32 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:41:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:41:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/rangeradmin_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/rangeradmin_127.8.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/rangerlookup_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/rangerlookup_127.8.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/HTTP_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/HTTP_127.8.241.212@KRBTEST.COM.keytab.
I20260806 18:41:32.461902 9159 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5RraP2/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/postgres -l logfile start
2026-08-06 18:41:34.713 UTC [27296] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:41:34.713 UTC [27296] LOG: listening on IPv4 address "127.8.241.212", port 48779
2026-08-06 18:41:34.716 UTC [27296] LOG: listening on Unix socket "/tmp/.s.PGSQL.48779"
2026-08-06 18:41:34.736 UTC [27300] LOG: database system was shut down at 2026-08-06 18:41:33 UTC
2026-08-06 18:41:34.740 UTC [27296] LOG: database system is ready to accept connections
I20260806 18:41:35.784070 9159 mini_postgres.cc:96] Postgres bound to 48779
2026-08-06 18:41:35.791 UTC [27313] FATAL: database "slave" does not exist
127.8.241.212:48779 - accepting connections
I20260806 18:41:35.791558 9159 mini_ranger.cc:161] Starting Ranger...
I20260806 18:41:35.806607 9159 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:41:36.215615 9159 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:41:36.215690 9159 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-admin
2026-08-06 18:41:36,485 [I] DB FLAVOR :POSTGRES
2026-08-06 18:41:36,486 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:41:36,486 [I] Checking connection..
2026-08-06 18:41:36,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:41:36,755 [I] Checking connection passed.
2026-08-06 18:41:36,756 [I] --------- Verifying version history table ---------
2026-08-06 18:41:36,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:41:37,021 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:41:37,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:41:37,265 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:41:37,265 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:41:37,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:41:37.529 UTC [27411] WARNING: there is no transaction in progress
2026-08-06 18:41:37,537 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:41:37,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:41:37,800 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:41:37,800 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:41:37,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:41:38,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:41:38,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-k5rr.c.gcp-upstream.internal','N') ;"
2026-08-06 18:41:38,543 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:41:38,804 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:41:38,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:41:39,068 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:41:39,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:41:39,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:41:39,586 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:41:39,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:41:40.498 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.514 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.529 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.543 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.557 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.574 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.627 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.633 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.645 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.655 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.664 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.677 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.687 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.693 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.699 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:40.705 UTC [27577] WARNING: there is no transaction in progress
2026-08-06 18:41:41,264 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:41:41,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:41:41,496 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:41:41,496 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:41:41,764 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:41:41,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:41:42,026 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:41:42,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:41:42,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:41:42,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-k5rr.c.gcp-upstream.internal';"
2026-08-06 18:41:42,806 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:41:42,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:41:43,051 [I] DB_PATCHES have already been applied
I20260806 18:41:43.057161 9159 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-admin:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5RraP2/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:41:43.057226 9159 mini_ranger.cc:193] Using host: 127.8.241.212
I20260806 18:41:43.061249 9159 mini_ranger.cc:227] Ranger admin URL: http://127.8.241.212:59477
Aug 06, 2026 6:41:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:41:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:41:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:41:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:41:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:41:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:41:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:41:44.254238 9159 mini_ranger.cc:160] Time spent starting Ranger: real 8.463s user 0.001s sys 0.009s
Aug 06, 2026 6:42:05 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:42:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:42:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:42:05 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:42:06 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:42:06.979784 9159 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.241.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.241.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/rangerkms_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.241.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/rangerkms_127.8.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.241.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.241.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.241.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/HTTP_127.8.241.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.241.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/HTTP_127.8.241.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:42:07 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041727, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:42:07.082262 9159 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:42:07.093518 9159 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:42:07.953699 9159 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:42:07.953785 9159 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-kms
I20260806 18:42:07.953800 9159 mini_ranger_kms.cc:227] Using postgres at 127.8.241.212:48779
2026-08-06 18:42:08,127 [I] DB FLAVOR :POSTGRES
2026-08-06 18:42:08,128 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:42:08,128 [I] Checking connection
2026-08-06 18:42:08,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:42:08,387 [I] connection success
2026-08-06 18:42:08,387 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:42:08,387 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:42:08,387 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:42:08,672 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:42:08,673 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:42:08,673 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:42:08,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.241.212:48779/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:42:08,999 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:42:09.462558 9159 mini_ranger_kms.cc:310] Created kms service
I20260806 18:42:09.680316 9159 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:42:09.726167 9159 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:42:09 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041727, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:10.143551 9159 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:42:10.143705 9159 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-kms:/tmp/dist-test-task5RraP2/build/debug/bin/postgresql.jar:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5RraP2/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:42:10.143736 9159 mini_ranger_kms.cc:240] Using host: 127.8.241.212
I20260806 18:42:10.149154 9159 mini_ranger_kms.cc:276] Ranger KMS PID: 27851
I20260806 18:42:10.149226 9159 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.241.212:35817
Aug 06, 2026 6:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:42:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 18:42:12 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041732, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.241.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:12.624823 9159 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.543s user 0.007s sys 0.150s
I20260806 18:42:12.624949 9159 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:42:12.625059 9159 mini_ranger_kms.cc:401] 127.8.241.212:35817/kms/v1/keys
Aug 06 18:42:12 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041732, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.241.212@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
Aug 06 18:42:13 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041732, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.241.212@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
Aug 06 18:42:15 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:42:15.674511 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37203
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:37203
--ranger_config_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:35817/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:15.796182 27918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:15.796495 27918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:15.796535 27918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:15.800755 27918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:15.800843 27918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:15.800863 27918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:15.800879 27918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:15.800894 27918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:15.806733 27918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.241.212:35817/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:37203
--ranger_config_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:15.807983 27918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:15.809137 27918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:15.818655 27926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:15.818638 27925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:15.818847 27918 server_base.cc:1034] running on GCE node
W20260806 18:42:15.818638 27929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:15.819706 27918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:15.820904 27918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:15.822103 27918 hybrid_clock.cc:648] HybridClock initialized: now 1786041735822079 us; error 38 us; skew 500 ppm
Aug 06 18:42:15 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041735, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:15.824939 27918 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:42:15.825951 27918 webserver.cc:460] Webserver started at http://127.8.241.254:43317/ using document root <none> and password file <none>
I20260806 18:42:15.826508 27918 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:15.826567 27918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:15.826769 27918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:42:15 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041735, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:15.885458 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "f3a44146089d42849e3c1e92b20c2998"
format_stamp: "Formatted at 2026-08-06 18:42:15 on dist-test-slave-k5rr"
server_key: "8e917119fd7f0335b531a5f1dcffcd39"
server_key_iv: "101579ee4a67c11c85e5a187ee417ad6"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:15.886461 27918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "f3a44146089d42849e3c1e92b20c2998"
format_stamp: "Formatted at 2026-08-06 18:42:15 on dist-test-slave-k5rr"
server_key: "8e917119fd7f0335b531a5f1dcffcd39"
server_key_iv: "101579ee4a67c11c85e5a187ee417ad6"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:15.892027 27918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 18:42:15.910398 27941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:15.912123 27918 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:42:15.912371 27918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "f3a44146089d42849e3c1e92b20c2998"
format_stamp: "Formatted at 2026-08-06 18:42:15 on dist-test-slave-k5rr"
server_key: "8e917119fd7f0335b531a5f1dcffcd39"
server_key_iv: "101579ee4a67c11c85e5a187ee417ad6"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:15.912608 27918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:15.956947 27918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:15.960453 27918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:15.960692 27918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:15.970373 28016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:37203 every 8 connection(s)
I20260806 18:42:15.970382 27918 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:37203
I20260806 18:42:15.971837 27918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:42:15.973996 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 27918
I20260806 18:42:15.974123 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:42:15.974488 9159 external_mini_cluster.cc:1428] Setting key a4bb5b33d755291f9f1b8fdbf6d5e713
I20260806 18:42:15.975620 28018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:15.982868 28018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Bootstrap starting.
I20260806 18:42:15.985529 28018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Neither blocks nor log segments found. Creating new log.
Aug 06 18:42:15 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:15.986524 28018 log.cc:824] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:15.989040 28018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: No bootstrap required, opened a new log
I20260806 18:42:15.992285 28023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:15.976055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.8.241.254:37203)
0806 18:42:15.976563 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:15.976592 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:15.976644 (+ 52us) server_negotiation.cc:407] Connection header received
0806 18:42:15.977636 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:15.977663 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:15.978124 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:15.979371 (+ 1247us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:15.979858 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:15.981421 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:15.982710 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:15.983404 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:15.983568 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:15.986851 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:15.986859 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:15.986880 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:15.986919 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:15.989166 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:15.989607 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:15.989612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:15.989617 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:15.989734 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:15.990022 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:15.990026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:15.990027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:15.990522 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:15.990707 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:15.991038 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 18:42:15.991517 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":140,"threads_started":1}
I20260806 18:42:15.993526 28018 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } }
I20260806 18:42:15.993855 28018 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:15.993952 28018 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a44146089d42849e3c1e92b20c2998, State: Initialized, Role: FOLLOWER
I20260806 18:42:15.994479 28018 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } }
I20260806 18:42:15.994686 28018 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:42:15.994781 28018 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:42:15.994889 28018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:15.996138 28018 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } }
I20260806 18:42:15.996546 28018 leader_election.cc:302] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3a44146089d42849e3c1e92b20c2998; no voters:
I20260806 18:42:15.996950 28018 leader_election.cc:288] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:42:15.997115 28025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:42:15.997431 28025 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 1 LEADER]: Becoming Leader. State: Replica: f3a44146089d42849e3c1e92b20c2998, State: Running, Role: LEADER
I20260806 18:42:15.997833 28025 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } }
I20260806 18:42:15.999311 28025 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a44146089d42849e3c1e92b20c2998. Latest consensus state: current_term: 1 leader_uuid: "f3a44146089d42849e3c1e92b20c2998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } } }
I20260806 18:42:15.999485 28018 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:15.999526 28025 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:15.999940 28025 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3a44146089d42849e3c1e92b20c2998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } } }
I20260806 18:42:16.000046 28025 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:16.002578 28031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:42:16.002897 28018 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:42:16.009052 28031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:42:16.017624 28031 catalog_manager.cc:1347] Generated new cluster ID: d5bce35fe328444191672cc5480589d5
I20260806 18:42:16.017731 28031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:42:16.038779 28031 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:42:16.040199 28031 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:42:16.049871 28031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Generated new TSK 0
I20260806 18:42:16.050917 28031 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:42:16.788745 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:35817/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37203
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:16.910460 28081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:16.910719 28081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:16.910779 28081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:16.915349 28081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:16.915409 28081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:16.915488 28081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:42:16.922580 28081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.241.212:35817/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:16.923802 28081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:16.925222 28081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:16.935314 28093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:16.935314 28097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:16.935886 28094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:16.936261 28081 server_base.cc:1034] running on GCE node
I20260806 18:42:16.936672 28081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:16.937387 28081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:16.938563 28081 hybrid_clock.cc:648] HybridClock initialized: now 1786041736938538 us; error 38 us; skew 500 ppm
Aug 06 18:42:16 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041736, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:16.942103 28081 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:42:16.943344 28081 webserver.cc:460] Webserver started at http://127.8.241.193:35193/ using document root <none> and password file <none>
I20260806 18:42:16.944054 28081 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:16.944139 28081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:16.944396 28081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:42:16 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041736, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:16.962029 28081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "ba0f6d1378464cf68113e4d74eec61d9"
format_stamp: "Formatted at 2026-08-06 18:42:16 on dist-test-slave-k5rr"
server_key: "a978deba7be27caefa5edd143632a361"
server_key_iv: "d49cd708052fc3042fb66e2dd6f5728b"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:16.962769 28081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "ba0f6d1378464cf68113e4d74eec61d9"
format_stamp: "Formatted at 2026-08-06 18:42:16 on dist-test-slave-k5rr"
server_key: "a978deba7be27caefa5edd143632a361"
server_key_iv: "d49cd708052fc3042fb66e2dd6f5728b"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:16.974545 28081 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.001s
I20260806 18:42:16.997177 28107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:16.999032 28081 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260806 18:42:16.999194 28081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "ba0f6d1378464cf68113e4d74eec61d9"
format_stamp: "Formatted at 2026-08-06 18:42:16 on dist-test-slave-k5rr"
server_key: "a978deba7be27caefa5edd143632a361"
server_key_iv: "d49cd708052fc3042fb66e2dd6f5728b"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:16.999357 28081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:17.040055 28081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:17.043989 28081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:17.044200 28081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:17.044961 28081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:17.046164 28081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:17.046232 28081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:17.046293 28081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:17.046316 28081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:17.057114 28081 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:44889
I20260806 18:42:17.058478 28081 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:42:17.061666 28285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:44889 every 8 connection(s)
I20260806 18:42:17.067992 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 28081
I20260806 18:42:17.068126 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:42:17.068431 9159 external_mini_cluster.cc:1428] Setting key 8352f49051c85684d074f73e1c18894b
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041736, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:17.078413 28289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:17.064104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37203 (local address 127.8.241.193:37607)
0806 18:42:17.064653 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:17.064710 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:17.065723 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:17.066270 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:17.066276 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:17.066552 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:17.067051 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:17.067064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.067942 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.067945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:17.069248 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:17.069261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.070010 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.070015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:17.070208 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:17.070875 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:17.070898 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:17.072494 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:17.074636 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:17.074643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:17.074662 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:17.075006 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:17.075307 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:17.075309 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:17.075311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:17.075504 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:17.075977 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:17.075984 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:17.076328 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 18:42:17.076600 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 18:42:17.076980 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":127,"threads_started":1}
I20260806 18:42:17.079510 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:17.065294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:37607 (local address 127.8.241.254:37203)
0806 18:42:17.065575 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:17.065580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:17.065597 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:42:17.065928 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:17.065931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:17.065991 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:17.066081 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:17.067197 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.067783 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.069385 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.069722 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.069827 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:17.072669 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:17.072672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:17.072674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:17.072699 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:17.074499 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:17.075117 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:17.075120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:17.075122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:17.075212 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:17.075607 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:17.075610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:17.075612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:17.075834 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:17.075920 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:17.079173 (+ 3253us) server_negotiation.cc:299] Negotiation successful
0806 18:42:17.079336 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
I20260806 18:42:17.080700 28286 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37203
I20260806 18:42:17.080956 28286 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:17.081517 28286 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:42:17.083343 27963 ts_manager.cc:194] Registered new tserver with Master: ba0f6d1378464cf68113e4d74eec61d9 (127.8.241.193:44889)
I20260806 18:42:17.084949 27963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:37607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:42:17.104812 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:35817/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37203
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:42:17.236524 28294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:17.236903 28294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:17.236972 28294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:17.241748 28294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:17.241812 28294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:17.241873 28294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:42:17.249291 28294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.241.212:35817/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:17.250751 28294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:17.252198 28294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:17.263412 28309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:17.263541 28306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:17.263454 28312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:17.264806 28294 server_base.cc:1034] running on GCE node
I20260806 18:42:17.265332 28294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:17.266153 28294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:17.267375 28294 hybrid_clock.cc:648] HybridClock initialized: now 1786041737267342 us; error 44 us; skew 500 ppm
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:17.271256 28294 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:42:17.272565 28294 webserver.cc:460] Webserver started at http://127.8.241.194:39837/ using document root <none> and password file <none>
I20260806 18:42:17.273540 28294 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:17.273659 28294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:17.274008 28294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:17.298956 28294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "d3e219ea16ce4a4ead9eab9bbf312c97"
format_stamp: "Formatted at 2026-08-06 18:42:17 on dist-test-slave-k5rr"
server_key: "dad47aef8adb4f8c9c5e133159d0bdc5"
server_key_iv: "d1c5bac1c19e8f38b061107023d351e0"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:17.299954 28294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "d3e219ea16ce4a4ead9eab9bbf312c97"
format_stamp: "Formatted at 2026-08-06 18:42:17 on dist-test-slave-k5rr"
server_key: "dad47aef8adb4f8c9c5e133159d0bdc5"
server_key_iv: "d1c5bac1c19e8f38b061107023d351e0"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:17.310590 28294 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260806 18:42:17.331934 28325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:17.334210 28294 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260806 18:42:17.334353 28294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "d3e219ea16ce4a4ead9eab9bbf312c97"
format_stamp: "Formatted at 2026-08-06 18:42:17 on dist-test-slave-k5rr"
server_key: "dad47aef8adb4f8c9c5e133159d0bdc5"
server_key_iv: "d1c5bac1c19e8f38b061107023d351e0"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:17.334497 28294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:17.356087 28294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:17.359658 28294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:17.359865 28294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:17.360562 28294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:17.361625 28294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:17.361697 28294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:17.361765 28294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:17.361791 28294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:17.372152 28294 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:34385
I20260806 18:42:17.372229 28497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:34385 every 8 connection(s)
I20260806 18:42:17.373368 28294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:42:17.377689 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 28294
I20260806 18:42:17.377830 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:42:17.378154 9159 external_mini_cluster.cc:1428] Setting key f0fe50c5a0f165a6b674391b73fa97ef
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:42:17.392776 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:17.375326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:57353 (local address 127.8.241.254:37203)
0806 18:42:17.375483 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:17.375487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:17.376469 (+ 982us) server_negotiation.cc:407] Connection header received
0806 18:42:17.377729 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:17.377733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:17.377796 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:17.377897 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:17.379343 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.380156 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.382057 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.382288 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.382371 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:17.388213 (+ 5842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:17.388218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:17.388221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:17.388255 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:17.390059 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:17.390759 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:17.390763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:17.390765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:17.390861 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:17.391303 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:17.391306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:17.391308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:17.391575 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:17.391669 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:17.392441 (+ 772us) server_negotiation.cc:299] Negotiation successful
0806 18:42:17.392590 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:42:17.394709 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:17.375726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37203 (local address 127.8.241.194:57353)
0806 18:42:17.376274 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:17.376339 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:17.377455 (+ 1116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:17.378099 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:17.378109 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:17.378456 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:17.379159 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:17.379182 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.380348 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.380352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:17.381912 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:17.381926 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.382462 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.382467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:17.382690 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:17.386113 (+ 3423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:17.386136 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:17.387984 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:17.390210 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:17.390218 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:17.390240 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:17.390607 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:17.390986 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:17.390989 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:17.390991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:17.391191 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:17.391753 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:17.391761 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:17.392129 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 18:42:17.393248 (+ 1119us) client_negotiation.cc:240] Negotiation successful
0806 18:42:17.393634 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":136,"threads_started":1}
I20260806 18:42:17.396348 28498 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37203
I20260806 18:42:17.396663 28498 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:17.397390 28498 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
I20260806 18:42:17.398562 27963 ts_manager.cc:194] Registered new tserver with Master: d3e219ea16ce4a4ead9eab9bbf312c97 (127.8.241.194:34385)
I20260806 18:42:17.399252 27963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:57353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:42:17.424307 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:35817/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37203
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
W20260806 18:42:17.594815 28508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:17.595153 28508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:17.595209 28508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:17.600888 28508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:17.600967 28508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:17.601208 28508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:42:17.609066 28508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.241.212:35817/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:17.610404 28508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:17.611840 28508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:17.625248 28516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:17.626075 28519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:17.626348 28508 server_base.cc:1034] running on GCE node
W20260806 18:42:17.629192 28517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:17.629714 28508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:17.630594 28508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:17.631778 28508 hybrid_clock.cc:648] HybridClock initialized: now 1786041737631711 us; error 83 us; skew 500 ppm
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:17.635274 28508 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:42:17.636371 28508 webserver.cc:460] Webserver started at http://127.8.241.195:42981/ using document root <none> and password file <none>
I20260806 18:42:17.637126 28508 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:17.637210 28508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:17.637439 28508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:17.652580 28508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "ea2e9b64f1c34d5ca556b910acd381a6"
format_stamp: "Formatted at 2026-08-06 18:42:17 on dist-test-slave-k5rr"
server_key: "685f9ec26bbc3793c53370ac01c405e4"
server_key_iv: "8321510993ee0cb580a8f689b0fe5282"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:17.653345 28508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "ea2e9b64f1c34d5ca556b910acd381a6"
format_stamp: "Formatted at 2026-08-06 18:42:17 on dist-test-slave-k5rr"
server_key: "685f9ec26bbc3793c53370ac01c405e4"
server_key_iv: "8321510993ee0cb580a8f689b0fe5282"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:17.661314 28508 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.000s
I20260806 18:42:17.681874 28528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:17.683560 28508 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.001s
I20260806 18:42:17.683704 28508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "ea2e9b64f1c34d5ca556b910acd381a6"
format_stamp: "Formatted at 2026-08-06 18:42:17 on dist-test-slave-k5rr"
server_key: "685f9ec26bbc3793c53370ac01c405e4"
server_key_iv: "8321510993ee0cb580a8f689b0fe5282"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:17.683859 28508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:17.702153 28508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:17.705708 28508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:17.705910 28508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:17.706550 28508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:17.707626 28508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:17.707685 28508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:17.707754 28508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:17.707782 28508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:17.718082 28508 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:38485
I20260806 18:42:17.718156 28703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:38485 every 8 connection(s)
I20260806 18:42:17.719439 28508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:42:17.725826 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 28508
I20260806 18:42:17.725962 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:42:17.726315 9159 external_mini_cluster.cc:1428] Setting key 4275b4e841961db9ef195a862bee2fce
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:17.737428 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:17.721635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:51213 (local address 127.8.241.254:37203)
0806 18:42:17.721777 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:17.721782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:17.727081 (+ 5299us) server_negotiation.cc:407] Connection header received
0806 18:42:17.728307 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:17.728311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:17.728364 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:17.728413 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:17.729851 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.730381 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.731508 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.731727 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.731815 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:17.734189 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:17.734192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:17.734194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:17.734219 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:17.735481 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:17.735961 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:17.735964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:17.735965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:17.736038 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:17.736366 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:17.736369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:17.736372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:17.736547 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:17.736634 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:17.737147 (+ 513us) server_negotiation.cc:299] Negotiation successful
0806 18:42:17.737266 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:42:17.738327 28706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:17.722188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37203 (local address 127.8.241.195:51213)
0806 18:42:17.726918 (+ 4730us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:17.726977 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:17.728062 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:17.728855 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:17.728862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:17.729197 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:17.729693 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:17.729707 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.730501 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.730504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:17.731373 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:17.731383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:17.731870 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:17.731874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:17.732043 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:17.732623 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:17.732643 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:17.734033 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:17.735589 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:17.735595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:17.735612 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:17.735858 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:17.736118 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:17.736121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:17.736123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:17.736270 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:17.736646 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:17.736653 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:17.736899 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 18:42:17.737089 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 18:42:17.737421 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4467,"thread_start_us":153,"threads_started":1}
I20260806 18:42:17.739607 28704 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37203
I20260806 18:42:17.739868 28704 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:17.740473 28704 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
I20260806 18:42:17.741536 27963 ts_manager.cc:194] Registered new tserver with Master: ea2e9b64f1c34d5ca556b910acd381a6 (127.8.241.195:38485)
I20260806 18:42:17.742061 27963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:51213
I20260806 18:42:17.752286 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:42:17.752897 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 28081
I20260806 18:42:17.764197 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 28294
I20260806 18:42:17.779263 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 28508
I20260806 18:42:17.795395 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 27918
I20260806 18:42:17.812510 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37203
--webserver_interface=127.8.241.254
--webserver_port=43317
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:37203
--ranger_config_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:35817/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.241.212:35817/kms with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:17.926805 28711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:17.927088 28711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:17.927143 28711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:17.931135 28711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:17.931195 28711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:17.931221 28711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:17.931242 28711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:17.931263 28711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:17.936707 28711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.241.212:35817/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:37203
--ranger_config_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=43317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:17.937813 28711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:17.938910 28711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:17.947587 28719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:17.947613 28722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:17.947640 28718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:17.947755 28711 server_base.cc:1034] running on GCE node
I20260806 18:42:17.948465 28711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:17.949615 28711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:17.950793 28711 hybrid_clock.cc:648] HybridClock initialized: now 1786041737950758 us; error 44 us; skew 500 ppm
Aug 06 18:42:17 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:17.953626 28711 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:42:17.954540 28711 webserver.cc:460] Webserver started at http://127.8.241.254:43317/ using document root <none> and password file <none>
I20260806 18:42:17.955089 28711 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:17.955158 28711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:18.038686 28711 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:42:18.041033 28736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.042440 28711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:42:18.042582 28711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "f3a44146089d42849e3c1e92b20c2998"
format_stamp: "Formatted at 2026-08-06 18:42:15 on dist-test-slave-k5rr"
server_key: "8e917119fd7f0335b531a5f1dcffcd39"
server_key_iv: "101579ee4a67c11c85e5a187ee417ad6"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:18.043188 28711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:18.069511 28711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:18.072422 28711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:18.072604 28711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:18.079818 28711 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:37203
I20260806 18:42:18.079843 28816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:37203 every 8 connection(s)
I20260806 18:42:18.080937 28711 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:42:18.085685 28818 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:42:18.087422 28818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Bootstrap starting.
I20260806 18:42:18.090494 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 28711
I20260806 18:42:18.090993 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:35817/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:44889
--local_ip_for_outbound_sockets=127.8.241.193
--tserver_master_addrs=127.8.241.254:37203
--webserver_port=35193
--webserver_interface=127.8.241.193
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.241.212:35817/kms with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:42:18.105481 28818 log.cc:824] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:18.112850 28818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:42:18.113284 28818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Bootstrap complete.
I20260806 18:42:18.117449 28818 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } }
I20260806 18:42:18.117866 28818 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3a44146089d42849e3c1e92b20c2998, State: Initialized, Role: FOLLOWER
I20260806 18:42:18.118397 28818 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } }
I20260806 18:42:18.118532 28818 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:42:18.118604 28818 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:42:18.118716 28818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:42:18.120651 28818 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } }
I20260806 18:42:18.120990 28818 leader_election.cc:302] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3a44146089d42849e3c1e92b20c2998; no voters:
I20260806 18:42:18.121374 28818 leader_election.cc:288] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:42:18.121466 28824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:42:18.121657 28824 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [term 2 LEADER]: Becoming Leader. State: Replica: f3a44146089d42849e3c1e92b20c2998, State: Running, Role: LEADER
I20260806 18:42:18.121976 28824 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } }
I20260806 18:42:18.122753 28818 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:18.123392 28826 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3a44146089d42849e3c1e92b20c2998. Latest consensus state: current_term: 2 leader_uuid: "f3a44146089d42849e3c1e92b20c2998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } } }
I20260806 18:42:18.123529 28826 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:18.123847 28825 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f3a44146089d42849e3c1e92b20c2998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3a44146089d42849e3c1e92b20c2998" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37203 } } }
I20260806 18:42:18.123944 28825 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:18.124256 28836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:42:18.127651 28836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:42:18.128216 28818 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:42:18.128352 28836 catalog_manager.cc:1259] Loaded cluster ID: d5bce35fe328444191672cc5480589d5
I20260806 18:42:18.128423 28836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:42:18.130874 28836 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:42:18.131640 28836 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Loaded TSK: 0
I20260806 18:42:18.132884 28836 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:42:18.237604 28821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:18.237950 28821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:18.238018 28821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:18.244042 28821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:18.244138 28821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:18.244238 28821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:42:18.252140 28821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.241.212:35817/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:44889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=35193
--tserver_master_addrs=127.8.241.254:37203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:18.253594 28821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:18.255013 28821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:18.275650 28872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:18.275683 28873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:18.276175 28875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:18.276754 28821 server_base.cc:1034] running on GCE node
I20260806 18:42:18.277294 28821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:18.278085 28821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:18.279264 28821 hybrid_clock.cc:648] HybridClock initialized: now 1786041738279242 us; error 42 us; skew 500 ppm
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041738, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:18.286131 28821 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:42:18.287415 28821 webserver.cc:460] Webserver started at http://127.8.241.193:35193/ using document root <none> and password file <none>
I20260806 18:42:18.288192 28821 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:18.288307 28821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041738, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:18.306849 28821 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260806 18:42:18.310123 28887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.311678 28821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:42:18.311802 28821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "ba0f6d1378464cf68113e4d74eec61d9"
format_stamp: "Formatted at 2026-08-06 18:42:16 on dist-test-slave-k5rr"
server_key: "a978deba7be27caefa5edd143632a361"
server_key_iv: "d49cd708052fc3042fb66e2dd6f5728b"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:18.312453 28821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:18.335736 28821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:18.339345 28821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:18.339552 28821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:18.340265 28821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:18.341403 28821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:18.341470 28821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.341531 28821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:18.341555 28821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.352540 28821 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:44889
I20260806 18:42:18.352598 29050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:44889 every 8 connection(s)
I20260806 18:42:18.353883 28821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:42:18.363281 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 28821
I20260806 18:42:18.363792 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:35817/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:34385
--local_ip_for_outbound_sockets=127.8.241.194
--tserver_master_addrs=127.8.241.254:37203
--webserver_port=39837
--webserver_interface=127.8.241.194
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.241.212:35817/kms with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041738, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:18.377216 29054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:18.358368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37203 (local address 127.8.241.193:60127)
0806 18:42:18.359145 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:18.359197 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:18.360043 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:18.361335 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:18.361342 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:18.361609 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:18.362155 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:18.362178 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.365775 (+ 3597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.365780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:18.367248 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:18.367262 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.368565 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.368569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:18.368779 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:18.369467 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:18.369485 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:18.370957 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:18.373408 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:18.373414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:18.373431 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:18.373780 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:18.374192 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:18.374195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:18.374197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:18.374378 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:18.375183 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:18.375190 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:18.375497 (+ 307us) client_negotiation.cc:769] Sending connection context
0806 18:42:18.375700 (+ 203us) client_negotiation.cc:240] Negotiation successful
0806 18:42:18.376058 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":142,"threads_started":1}
I20260806 18:42:18.379460 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:18.358201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:60127 (local address 127.8.241.254:37203)
0806 18:42:18.358854 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:18.358878 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:18.359334 (+ 456us) server_negotiation.cc:407] Connection header received
0806 18:42:18.360478 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:18.360488 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:18.360856 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:18.361270 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:18.362345 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.363890 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.367406 (+ 3516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.368165 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.368327 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:18.371153 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:18.371160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:18.371180 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:18.371213 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:18.373268 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:18.373920 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:18.373927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:18.373933 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:18.374078 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:18.374488 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:18.374492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:18.374493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:18.374996 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:18.375126 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:18.378175 (+ 3049us) server_negotiation.cc:299] Negotiation successful
0806 18:42:18.378505 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":118,"threads_started":1}
I20260806 18:42:18.382342 29051 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37203
I20260806 18:42:18.382622 29051 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:18.383301 29051 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
W20260806 18:42:18.385452 29051 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:37203 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:42:18.385569 29051 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:18.385752 29051 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
I20260806 18:42:18.386543 29051 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:18.386713 29051 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
W20260806 18:42:18.387416 29051 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:42:18.542526 29056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:18.542906 29056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:18.542994 29056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:18.548497 29056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:18.548583 29056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:18.548674 29056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:42:18.556674 29056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.241.212:35817/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:34385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=39837
--tserver_master_addrs=127.8.241.254:37203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:18.558159 29056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:18.559499 29056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:18.572512 29073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:18.572933 29069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:18.572527 29067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:18.577278 29056 server_base.cc:1034] running on GCE node
I20260806 18:42:18.577877 29056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:18.578811 29056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:18.579998 29056 hybrid_clock.cc:648] HybridClock initialized: now 1786041738579921 us; error 100 us; skew 500 ppm
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041738, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:18.583895 29056 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:42:18.585259 29056 webserver.cc:460] Webserver started at http://127.8.241.194:39837/ using document root <none> and password file <none>
I20260806 18:42:18.586107 29056 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:18.586298 29056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041738, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:18.613286 29056 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 18:42:18.616711 29084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.618628 29056 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:42:18.618788 29056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "d3e219ea16ce4a4ead9eab9bbf312c97"
format_stamp: "Formatted at 2026-08-06 18:42:17 on dist-test-slave-k5rr"
server_key: "dad47aef8adb4f8c9c5e133159d0bdc5"
server_key_iv: "d1c5bac1c19e8f38b061107023d351e0"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:18.619390 29056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:18.635198 29056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:18.639658 29056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:18.639882 29056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:18.640697 29056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:18.641952 29056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:18.642023 29056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.642092 29056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:18.642127 29056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.653779 29056 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:34385
I20260806 18:42:18.655220 29056 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:42:18.655866 29247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:34385 every 8 connection(s)
I20260806 18:42:18.662539 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 29056
I20260806 18:42:18.663086 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.241.212:35817/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:38485
--local_ip_for_outbound_sockets=127.8.241.195
--tserver_master_addrs=127.8.241.254:37203
--webserver_port=42981
--webserver_interface=127.8.241.195
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.241.212:35817/kms with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041738, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:18.672444 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:18.657568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:57471 (local address 127.8.241.254:37203)
0806 18:42:18.658073 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:18.658079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:18.658803 (+ 724us) server_negotiation.cc:407] Connection header received
0806 18:42:18.659927 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:18.659930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:18.660001 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:18.660085 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:18.661662 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.662378 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.664763 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.665152 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.665251 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:18.668114 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:18.668119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:18.668122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:18.668156 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:18.669676 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:18.670367 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:18.670371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:18.670373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:18.670483 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:18.670961 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:18.670965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:18.670967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:18.671205 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:18.671287 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:18.672098 (+ 811us) server_negotiation.cc:299] Negotiation successful
0806 18:42:18.672255 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359}
I20260806 18:42:18.674444 29254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:18.658130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37203 (local address 127.8.241.194:57471)
0806 18:42:18.658638 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:18.658691 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:18.659689 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:18.660297 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:18.660308 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:18.660677 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:18.661437 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:18.661464 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.662573 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.662576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:18.664001 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:18.664016 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.665466 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.665469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:18.665625 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:18.666307 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:18.666326 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:18.667887 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:18.669827 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:18.669834 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:18.669852 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:18.670223 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:18.670616 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:18.670619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:18.670621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:18.670827 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:18.671380 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:18.671388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:18.671761 (+ 373us) client_negotiation.cc:769] Sending connection context
0806 18:42:18.672894 (+ 1133us) client_negotiation.cc:240] Negotiation successful
0806 18:42:18.673309 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":126,"threads_started":1}
I20260806 18:42:18.676112 29248 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37203
I20260806 18:42:18.676466 29248 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:18.677245 29248 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
W20260806 18:42:18.679224 29248 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:37203 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:42:18.679425 29248 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:18.679672 29248 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
I20260806 18:42:18.680727 29248 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:18.680933 29248 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
W20260806 18:42:18.681851 29248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:42:18.803251 29255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:18.803493 29255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:18.803535 29255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:18.807570 29255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:18.807631 29255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:18.807721 29255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:42:18.813271 29255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.241.212:35817/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:38485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=42981
--tserver_master_addrs=127.8.241.254:37203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:18.814375 29255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:18.815517 29255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:18.828027 29264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:18.828027 29267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:18.828123 29265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:18.828217 29255 server_base.cc:1034] running on GCE node
I20260806 18:42:18.828807 29255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:18.829495 29255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:18.830674 29255 hybrid_clock.cc:648] HybridClock initialized: now 1786041738830651 us; error 34 us; skew 500 ppm
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041738, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:18.833606 29255 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:42:18.834640 29255 webserver.cc:460] Webserver started at http://127.8.241.195:42981/ using document root <none> and password file <none>
I20260806 18:42:18.835333 29255 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:18.835459 29255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041738, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:18.852160 29255 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:42:18.854692 29276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.856037 29255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:42:18.856154 29255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "ea2e9b64f1c34d5ca556b910acd381a6"
format_stamp: "Formatted at 2026-08-06 18:42:17 on dist-test-slave-k5rr"
server_key: "685f9ec26bbc3793c53370ac01c405e4"
server_key_iv: "8321510993ee0cb580a8f689b0fe5282"
server_key_version: "kuduclusterkey@0"
I20260806 18:42:18.856706 29255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:18.883407 29255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:18.886608 29255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:18.886809 29255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:18.887447 29255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:18.888371 29255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:18.888423 29255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.888478 29255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:18.888507 29255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:18.898823 29255 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:38485
I20260806 18:42:18.898859 29447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:38485 every 8 connection(s)
I20260806 18:42:18.900060 29255 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:42:18.906133 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 29255
Aug 06 18:42:18 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041738, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:18.919050 29453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:18.902684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37203 (local address 127.8.241.195:35867)
0806 18:42:18.904392 (+ 1708us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:18.904460 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:18.905664 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:18.906438 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:18.906448 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:18.906981 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:18.907727 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:18.907751 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.908815 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.908819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:18.910262 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:18.910274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.911016 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.911020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:18.911215 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:18.912157 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:18.912178 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:18.913828 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:18.915861 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:18.915866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:18.915884 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:18.916152 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:18.916457 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:18.916460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:18.916462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:18.916654 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:18.917059 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:18.917067 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:18.917396 (+ 329us) client_negotiation.cc:769] Sending connection context
0806 18:42:18.917609 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 18:42:18.917974 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1443,"thread_start_us":152,"threads_started":1}
I20260806 18:42:18.919957 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:18.903913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:35867 (local address 127.8.241.254:37203)
0806 18:42:18.904066 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:18.904072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:18.904583 (+ 511us) server_negotiation.cc:407] Connection header received
0806 18:42:18.905953 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:18.905957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:18.906020 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:18.906119 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:18.907919 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.908663 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.910415 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.910742 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.910816 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:18.914047 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:18.914051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:18.914054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:18.914078 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:18.915733 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:18.916252 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:18.916256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:18.916258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:18.916355 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:18.916753 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:18.916755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:18.916756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:18.916942 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:18.917075 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:18.919650 (+ 2575us) server_negotiation.cc:299] Negotiation successful
0806 18:42:18.919785 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61,"spinlock_wait_cycles":768}
I20260806 18:42:18.921058 29448 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37203
I20260806 18:42:18.921334 29448 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:18.921936 29448 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
I20260806 18:42:18.923122 29456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:18.906805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.8.241.254:37203)
0806 18:42:18.907370 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:18.907373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:18.907383 (+ 10us) server_negotiation.cc:407] Connection header received
0806 18:42:18.907415 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:18.907417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:18.907450 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:18.907524 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:18.908324 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.909128 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.911279 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:18.911638 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:18.911750 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:18.914171 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:18.914175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:18.914178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:18.914216 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:18.915848 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:18.916308 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:18.916311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:18.916312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:18.916389 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:18.916972 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:18.916975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:18.916977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:18.917184 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:18.917310 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:18.922868 (+ 5558us) server_negotiation.cc:299] Negotiation successful
0806 18:42:18.922999 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":89,"threads_started":1}
I20260806 18:42:18.923506 28761 ts_manager.cc:194] Registered new tserver with Master: ea2e9b64f1c34d5ca556b910acd381a6 (127.8.241.195:38485)
I20260806 18:42:18.925228 28761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:35867
Aug 06 18:42:19 dist-test-slave-k5rr krb5kdc[27252](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041739, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:19.387668 29051 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:19.388053 29051 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
I20260806 18:42:19.389084 28761 ts_manager.cc:194] Registered new tserver with Master: ba0f6d1378464cf68113e4d74eec61d9 (127.8.241.193:44889)
I20260806 18:42:19.389923 28761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:60127
Aug 06 18:42:19 dist-test-slave-k5rr krb5kdc[27252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041739, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for HTTP/127.8.241.212@KRBTEST.COM
I20260806 18:42:19.682108 29248 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:19.682451 29248 heartbeater.cc:511] Master 127.8.241.254:37203 requested a full tablet report, sending...
I20260806 18:42:19.683588 28761 ts_manager.cc:194] Registered new tserver with Master: d3e219ea16ce4a4ead9eab9bbf312c97 (127.8.241.194:34385)
I20260806 18:42:19.684391 28761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:57471
I20260806 18:42:19.689082 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:42:19.703177 29481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.691288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.8.241.254:37203)
0806 18:42:19.691915 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:19.691920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:19.691932 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:42:19.691977 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:19.691981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:19.692031 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:19.692117 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:19.693619 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.694967 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.696207 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.696578 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.696678 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.698496 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:19.698500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:19.698502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:19.698533 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:19.700338 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:19.700846 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:19.700849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:19.700851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:19.700944 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:19.701286 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:19.701290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:19.701293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:19.701550 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:19.701670 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:19.702505 (+ 835us) server_negotiation.cc:299] Negotiation successful
0806 18:42:19.702995 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":94,"threads_started":1}
I20260806 18:42:19.707221 28761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:42:19.711560 28761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:42:19.740993 29492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.728410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44889 (local address 127.0.0.1:56104)
0806 18:42:19.733615 (+ 5205us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:19.733629 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:19.733717 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:19.734643 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:19.734651 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:19.734678 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:19.734830 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.734839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.736632 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.736635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.738090 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.738102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.740460 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.740464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.740568 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.740609 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:42:19.740714 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 18:42:19.740830 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5139,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260806 18:42:19.742261 29497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.729137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56104 (local address 127.8.241.193:44889)
0806 18:42:19.733723 (+ 4586us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:19.733730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:19.733782 (+ 52us) server_negotiation.cc:407] Connection header received
0806 18:42:19.733867 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:19.733872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:19.734103 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:19.734269 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:19.734993 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.736249 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.738234 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.739077 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.739195 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.739364 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:19.742010 (+ 2646us) server_negotiation.cc:299] Negotiation successful
0806 18:42:19.742116 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4470,"thread_start_us":79,"threads_started":1}
I20260806 18:42:19.742866 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.728680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:34385 (local address 127.0.0.1:53512)
0806 18:42:19.734785 (+ 6105us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:19.734798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:19.734889 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:19.735578 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:19.735582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:19.735601 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:19.735708 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.735715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.737872 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.737875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.740493 (+ 2618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.740505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.742495 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.742499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.742581 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.742603 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:42:19.742668 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:42:19.742733 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6037,"mutex_wait_us":1375,"thread_start_us":100,"threads_started":1}
I20260806 18:42:19.743341 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.729124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53512 (local address 127.8.241.194:34385)
0806 18:42:19.734560 (+ 5436us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:19.734566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:19.734984 (+ 418us) server_negotiation.cc:407] Connection header received
0806 18:42:19.735066 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:19.735071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:19.735274 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:19.735427 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:19.735838 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.737748 (+ 1910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.740649 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.741510 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.741623 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.741773 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:19.743087 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0806 18:42:19.743203 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5328,"thread_start_us":124,"threads_started":1}
I20260806 18:42:19.744599 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.727224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:38485 (local address 127.0.0.1:32768)
0806 18:42:19.733142 (+ 5918us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:19.733200 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:19.733346 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:19.736388 (+ 3042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:19.736391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:19.736412 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:19.736526 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.736532 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.739291 (+ 2759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.739295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.741846 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.741857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.744200 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.744204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.744311 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.744334 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:42:19.744405 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:42:19.744468 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5687,"mutex_wait_us":1404,"thread_start_us":110,"threads_started":1}
I20260806 18:42:19.745028 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.727331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.8.241.195:38485)
0806 18:42:19.733955 (+ 6624us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:19.733962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:19.733986 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:42:19.734065 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:19.734070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:19.734277 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:19.734439 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:19.738090 (+ 3651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.739152 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.741987 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.743899 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.744006 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.744145 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:19.744777 (+ 632us) server_negotiation.cc:299] Negotiation successful
0806 18:42:19.744875 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6510,"thread_start_us":108,"threads_started":1}
I20260806 18:42:19.745767 28960 tablet_service.cc:1467] Processing CreateTablet for tablet 3b47f4d1b59140ea9ec34bdb760518b2 (DEFAULT_TABLE table=test-table [id=00d0f4871bd6467d98ea67d89db4cd3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:19.746186 29152 tablet_service.cc:1467] Processing CreateTablet for tablet 3b47f4d1b59140ea9ec34bdb760518b2 (DEFAULT_TABLE table=test-table [id=00d0f4871bd6467d98ea67d89db4cd3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:19.747046 28960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b47f4d1b59140ea9ec34bdb760518b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:19.747238 29152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b47f4d1b59140ea9ec34bdb760518b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:19.751886 29349 tablet_service.cc:1467] Processing CreateTablet for tablet 3b47f4d1b59140ea9ec34bdb760518b2 (DEFAULT_TABLE table=test-table [id=00d0f4871bd6467d98ea67d89db4cd3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:19.753044 29349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b47f4d1b59140ea9ec34bdb760518b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:19.762656 29504 tablet_bootstrap.cc:492] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9: Bootstrap starting.
I20260806 18:42:19.763350 29499 tablet_bootstrap.cc:492] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: Bootstrap starting.
I20260806 18:42:19.764161 29500 tablet_bootstrap.cc:492] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: Bootstrap starting.
I20260806 18:42:19.765292 29504 tablet_bootstrap.cc:654] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:19.766192 29504 log.cc:824] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:19.766400 29500 tablet_bootstrap.cc:654] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:19.767246 29500 log.cc:824] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:19.768456 29504 tablet_bootstrap.cc:492] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9: No bootstrap required, opened a new log
I20260806 18:42:19.768668 29504 ts_tablet_manager.cc:1397] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:42:19.769362 29500 tablet_bootstrap.cc:492] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: No bootstrap required, opened a new log
I20260806 18:42:19.769572 29500 ts_tablet_manager.cc:1397] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:42:19.772990 29504 raft_consensus.cc:348] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } }
I20260806 18:42:19.773299 29504 raft_consensus.cc:374] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:19.773357 29504 raft_consensus.cc:729] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0f6d1378464cf68113e4d74eec61d9, State: Initialized, Role: FOLLOWER
I20260806 18:42:19.773962 29500 raft_consensus.cc:348] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } }
I20260806 18:42:19.774097 29504 consensus_queue.cc:260] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } }
I20260806 18:42:19.774215 29500 raft_consensus.cc:374] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:19.774271 29500 raft_consensus.cc:729] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2e9b64f1c34d5ca556b910acd381a6, State: Initialized, Role: FOLLOWER
I20260806 18:42:19.774832 29500 consensus_queue.cc:260] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } }
I20260806 18:42:19.775185 29504 ts_tablet_manager.cc:1428] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260806 18:42:19.775554 29051 heartbeater.cc:503] Master 127.8.241.254:37203 was elected leader, sending a full tablet report...
I20260806 18:42:19.775818 29500 ts_tablet_manager.cc:1428] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:42:19.776260 29448 heartbeater.cc:503] Master 127.8.241.254:37203 was elected leader, sending a full tablet report...
I20260806 18:42:19.778098 29499 tablet_bootstrap.cc:654] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:19.778968 29499 log.cc:824] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:19.781100 29499 tablet_bootstrap.cc:492] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: No bootstrap required, opened a new log
I20260806 18:42:19.781299 29499 ts_tablet_manager.cc:1397] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260806 18:42:19.785722 29499 raft_consensus.cc:348] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } }
I20260806 18:42:19.786024 29499 raft_consensus.cc:374] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:19.786089 29499 raft_consensus.cc:729] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3e219ea16ce4a4ead9eab9bbf312c97, State: Initialized, Role: FOLLOWER
I20260806 18:42:19.786572 29499 consensus_queue.cc:260] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } }
I20260806 18:42:19.787654 29499 ts_tablet_manager.cc:1428] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:42:19.788031 29248 heartbeater.cc:503] Master 127.8.241.254:37203 was elected leader, sending a full tablet report...
I20260806 18:42:19.795836 29512 raft_consensus.cc:482] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:19.796020 29512 raft_consensus.cc:504] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } }
I20260806 18:42:19.797142 29512 leader_election.cc:288] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea2e9b64f1c34d5ca556b910acd381a6 (127.8.241.195:38485), ba0f6d1378464cf68113e4d74eec61d9 (127.8.241.193:44889)
I20260806 18:42:19.802481 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.797364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:38485 (local address 127.8.241.194:45879)
0806 18:42:19.797928 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:19.797942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:19.798060 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:19.798365 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:19.798369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:19.798384 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:19.798524 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.798531 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.799560 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.799563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.800665 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.800675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.802042 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.802046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.802143 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.802170 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:42:19.802243 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:42:19.802312 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":125,"threads_started":1}
I20260806 18:42:19.802510 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.797729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:45879 (local address 127.8.241.195:38485)
0806 18:42:19.797885 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:19.797890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:19.798023 (+ 133us) server_negotiation.cc:407] Connection header received
0806 18:42:19.798236 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:19.798240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:19.798281 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:19.798341 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:19.798652 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.799440 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.800784 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.801388 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.801474 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.801517 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:19.802292 (+ 775us) server_negotiation.cc:299] Negotiation successful
0806 18:42:19.802382 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:42:19.803437 29497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.797528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:45951 (local address 127.8.241.193:44889)
0806 18:42:19.797769 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:19.797774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:19.798115 (+ 341us) server_negotiation.cc:407] Connection header received
0806 18:42:19.798221 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:19.798224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:19.798278 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:19.798358 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:19.798686 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.799192 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.800669 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.801291 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.801385 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.801459 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:19.803222 (+ 1763us) server_negotiation.cc:299] Negotiation successful
0806 18:42:19.803312 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260806 18:42:19.803452 29375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b47f4d1b59140ea9ec34bdb760518b2" candidate_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" is_pre_election: true
I20260806 18:42:19.803777 29375 raft_consensus.cc:2454] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3e219ea16ce4a4ead9eab9bbf312c97 in term 0.
I20260806 18:42:19.804100 29514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.797379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44889 (local address 127.8.241.194:45951)
0806 18:42:19.798036 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:19.798048 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:19.798132 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:19.798478 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:19.798481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:19.798496 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:19.798589 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.798595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.799304 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.799308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.800518 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:19.800529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.802988 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.802992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:19.803087 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.803108 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:42:19.803811 (+ 703us) client_negotiation.cc:240] Negotiation successful
0806 18:42:19.803972 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":129,"threads_started":1}
I20260806 18:42:19.804320 29090 leader_election.cc:302] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3e219ea16ce4a4ead9eab9bbf312c97, ea2e9b64f1c34d5ca556b910acd381a6; no voters:
I20260806 18:42:19.804755 29512 raft_consensus.cc:2781] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:42:19.804903 29512 raft_consensus.cc:482] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:42:19.804852 28990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b47f4d1b59140ea9ec34bdb760518b2" candidate_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0f6d1378464cf68113e4d74eec61d9" is_pre_election: true
I20260806 18:42:19.805178 28990 raft_consensus.cc:2454] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3e219ea16ce4a4ead9eab9bbf312c97 in term 0.
I20260806 18:42:19.805141 29512 raft_consensus.cc:3037] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:19.806711 29512 raft_consensus.cc:504] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } }
I20260806 18:42:19.807070 29512 leader_election.cc:288] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [CANDIDATE]: Term 1 election: Requested vote from peers ea2e9b64f1c34d5ca556b910acd381a6 (127.8.241.195:38485), ba0f6d1378464cf68113e4d74eec61d9 (127.8.241.193:44889)
I20260806 18:42:19.807432 29375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b47f4d1b59140ea9ec34bdb760518b2" candidate_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea2e9b64f1c34d5ca556b910acd381a6"
I20260806 18:42:19.807564 29375 raft_consensus.cc:3037] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:19.807682 28990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b47f4d1b59140ea9ec34bdb760518b2" candidate_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0f6d1378464cf68113e4d74eec61d9"
I20260806 18:42:19.807790 28990 raft_consensus.cc:3037] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:19.808872 29375 raft_consensus.cc:2454] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3e219ea16ce4a4ead9eab9bbf312c97 in term 1.
I20260806 18:42:19.808969 28990 raft_consensus.cc:2454] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3e219ea16ce4a4ead9eab9bbf312c97 in term 1.
I20260806 18:42:19.809303 29087 leader_election.cc:302] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba0f6d1378464cf68113e4d74eec61d9, d3e219ea16ce4a4ead9eab9bbf312c97; no voters:
I20260806 18:42:19.809521 29512 raft_consensus.cc:2781] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:42:19.809711 29512 raft_consensus.cc:686] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 1 LEADER]: Becoming Leader. State: Replica: d3e219ea16ce4a4ead9eab9bbf312c97, State: Running, Role: LEADER
I20260806 18:42:19.810066 29512 consensus_queue.cc:237] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } }
I20260806 18:42:19.813516 28760 catalog_manager.cc:5495] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3e219ea16ce4a4ead9eab9bbf312c97 (127.8.241.194). New cstate: current_term: 1 leader_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3e219ea16ce4a4ead9eab9bbf312c97" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 34385 } health_report { overall_health: HEALTHY } } }
I20260806 18:42:19.831219 29498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.826549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53524 (local address 127.8.241.194:34385)
0806 18:42:19.826658 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:19.826663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:19.826678 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:42:19.826754 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:19.826756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:19.826808 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:19.826895 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:42:19.827250 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.828036 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.829330 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:19.829677 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:19.829789 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:19.830153 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:42:19.830753 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:42:19.830884 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:19.830987 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 18:42:19.831056 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:42:19.838770 28990 raft_consensus.cc:1261] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9 [term 1 FOLLOWER]: Refusing update from remote peer d3e219ea16ce4a4ead9eab9bbf312c97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:19.838899 29375 raft_consensus.cc:1261] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [term 1 FOLLOWER]: Refusing update from remote peer d3e219ea16ce4a4ead9eab9bbf312c97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:19.839664 29517 consensus_queue.cc:1035] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0f6d1378464cf68113e4d74eec61d9" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 44889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:19.840220 29517 consensus_queue.cc:1035] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2e9b64f1c34d5ca556b910acd381a6" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 38485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:19.851675 29522 mvcc.cc:205] Tried to move back new op lower bound from 7315626966371409920 to 7315626966264979456. Current Snapshot: MvccSnapshot[applied={T|T < 7315626966371409920}]
I20260806 18:42:19.854516 29526 mvcc.cc:205] Tried to move back new op lower bound from 7315626966371409920 to 7315626966264979456. Current Snapshot: MvccSnapshot[applied={T|T < 7315626966371409920}]
W20260806 18:42:19.858863 29052 tablet.cc:2369] T 3b47f4d1b59140ea9ec34bdb760518b2 P ba0f6d1378464cf68113e4d74eec61d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:42:19.863533 29525 mvcc.cc:205] Tried to move back new op lower bound from 7315626966371409920 to 7315626966264979456. Current Snapshot: MvccSnapshot[applied={T|T < 7315626966371409920}]
I20260806 18:42:19.864629 28760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:42:19.864800 28760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:42:19.867569 28760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3a44146089d42849e3c1e92b20c2998: Sending DeleteTablet for 3 replicas of tablet 3b47f4d1b59140ea9ec34bdb760518b2
I20260806 18:42:19.868443 29349 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b47f4d1b59140ea9ec34bdb760518b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:19 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:32768
I20260806 18:42:19.870471 28960 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b47f4d1b59140ea9ec34bdb760518b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:19 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:56104
I20260806 18:42:19.871174 29152 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b47f4d1b59140ea9ec34bdb760518b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:19 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:53512
I20260806 18:42:19.872216 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 28821
I20260806 18:42:19.889492 29540 tablet_replica.cc:330] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: stopping tablet replica
I20260806 18:42:19.894686 29540 raft_consensus.cc:2229] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:42:19.895306 29536 tablet_replica.cc:330] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: stopping tablet replica
I20260806 18:42:19.895911 29540 raft_consensus.cc:2258] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:42:19.896373 29536 raft_consensus.cc:2229] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:42:19.896803 29536 raft_consensus.cc:2258] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:42:19.898977 29540 ts_tablet_manager.cc:1905] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:42:19.905504 29540 ts_tablet_manager.cc:1918] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:42:19.905682 29540 log.cc:1196] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/3b47f4d1b59140ea9ec34bdb760518b2
I20260806 18:42:19.906100 29540 ts_tablet_manager.cc:1939] T 3b47f4d1b59140ea9ec34bdb760518b2 P d3e219ea16ce4a4ead9eab9bbf312c97: Deleting consensus metadata
I20260806 18:42:19.907492 28739 catalog_manager.cc:4841] TS d3e219ea16ce4a4ead9eab9bbf312c97 (127.8.241.194:34385): tablet 3b47f4d1b59140ea9ec34bdb760518b2 (table test-table [id=00d0f4871bd6467d98ea67d89db4cd3b]) successfully deleted
W20260806 18:42:19.908072 28737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:44889 (error 108)
I20260806 18:42:19.909186 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 29056
I20260806 18:42:19.912242 29536 ts_tablet_manager.cc:1905] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:42:19.914757 29536 ts_tablet_manager.cc:1918] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:42:19.914844 29536 log.cc:1196] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/3b47f4d1b59140ea9ec34bdb760518b2
I20260806 18:42:19.915091 29536 ts_tablet_manager.cc:1939] T 3b47f4d1b59140ea9ec34bdb760518b2 P ea2e9b64f1c34d5ca556b910acd381a6: Deleting consensus metadata
I20260806 18:42:19.945966 28740 catalog_manager.cc:4841] TS ea2e9b64f1c34d5ca556b910acd381a6 (127.8.241.195:38485): tablet 3b47f4d1b59140ea9ec34bdb760518b2 (table test-table [id=00d0f4871bd6467d98ea67d89db4cd3b]) successfully deleted
I20260806 18:42:19.948670 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:19.948243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:44889 (local address 127.0.0.1:56120)
0806 18:42:19.948466 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:19.948564 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260806 18:42:19.949033 28737 catalog_manager.cc:4584] TS ba0f6d1378464cf68113e4d74eec61d9 (127.8.241.193:44889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b47f4d1b59140ea9ec34bdb760518b2: Network error: Client connection negotiation failed: client connection to 127.8.241.193:44889: connect: Connection refused (error 111)
I20260806 18:42:19.956966 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 29255
I20260806 18:42:19.970165 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 28711
I20260806 18:42:19.984113 9159 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:42:20.328338 9159 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:42:20.328460 9159 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:42:20.672533 9159 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:42:20.672 UTC [27296] LOG: received smart shutdown request
2026-08-06 18:42:20.677 UTC [27296] LOG: background worker "logical replication launcher" (PID 27308) exited with exit code 1
2026-08-06 18:42:20.677 UTC [27301] LOG: shutting down
2026-08-06 18:42:20.819 UTC [27296] LOG: database system is shut down
2026-08-06T18:42:20Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49778 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:42:20 dist-test-slave-k5rr krb5kdc[29557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:42:20 dist-test-slave-k5rr krb5kdc[29557](info): set up 2 sockets
Aug 06 18:42:20 dist-test-slave-k5rr krb5kdc[29557](info): commencing operation
krb5kdc: starting...
W20260806 18:42:22.057744 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:42:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:42:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:42:22.177368 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42957
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:42957,127.8.241.253:44311,127.8.241.252:36847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:22.294248 29576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:22.294528 29576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:22.294585 29576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:22.298576 29576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:22.298632 29576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:22.298659 29576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:22.298682 29576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:22.298718 29576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:22.303982 29576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:42957,127.8.241.253:44311,127.8.241.252:36847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:22.305064 29576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:22.306207 29576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:22.314646 29585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:22.314659 29587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:22.314714 29576 server_base.cc:1034] running on GCE node
W20260806 18:42:22.314656 29589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:22.315452 29576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:22.316591 29576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:22.317742 29576 hybrid_clock.cc:648] HybridClock initialized: now 1786041742317726 us; error 32 us; skew 500 ppm
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:22.323326 29576 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:42:22.324223 29576 webserver.cc:460] Webserver started at http://127.8.241.254:34865/ using document root <none> and password file <none>
I20260806 18:42:22.324764 29576 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:22.324821 29576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:22.324993 29576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:22.327075 29576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "94200570712046f49d5c9e4a10cbcc38"
format_stamp: "Formatted at 2026-08-06 18:42:22 on dist-test-slave-k5rr"
server_key: "c940aacb37cf4ee8915d6312b3f73746"
server_key_iv: "e1840cb20aeea3a5491a8b25fcc2b1c1"
server_key_version: "encryptionkey@0"
I20260806 18:42:22.327605 29576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "94200570712046f49d5c9e4a10cbcc38"
format_stamp: "Formatted at 2026-08-06 18:42:22 on dist-test-slave-k5rr"
server_key: "c940aacb37cf4ee8915d6312b3f73746"
server_key_iv: "e1840cb20aeea3a5491a8b25fcc2b1c1"
server_key_version: "encryptionkey@0"
I20260806 18:42:22.331288 29576 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:42:22.333766 29601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:22.334996 29576 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:42:22.335099 29576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "94200570712046f49d5c9e4a10cbcc38"
format_stamp: "Formatted at 2026-08-06 18:42:22 on dist-test-slave-k5rr"
server_key: "c940aacb37cf4ee8915d6312b3f73746"
server_key_iv: "e1840cb20aeea3a5491a8b25fcc2b1c1"
server_key_version: "encryptionkey@0"
I20260806 18:42:22.335186 29576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:22.356300 29576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:22.359673 29576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:22.359840 29576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:22.367223 29576 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:42957
I20260806 18:42:22.367236 29672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:42957 every 8 connection(s)
I20260806 18:42:22.368193 29576 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:42:22.371104 29675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:22.373905 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 29576
I20260806 18:42:22.374032 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:42:22.374267 9159 external_mini_cluster.cc:1428] Setting key e36a80e11de564c2bb77493899dd1d6c
I20260806 18:42:22.375101 29675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.241.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.253@KRBTEST.COM" created.
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:22.397848 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.377623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42957 (local address 127.0.0.1:49742)
0806 18:42:22.378205 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.378244 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:22.379093 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:22.379818 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:22.379824 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:22.380274 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:22.380566 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.380581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.381778 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.381781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.382521 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.382531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.383286 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.383288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.383385 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.383912 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:22.383929 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:22.386475 (+ 2546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:22.394701 (+ 8226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.394712 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.394722 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.394989 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.395372 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.395374 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.395375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.395493 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.396178 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:22.396184 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:22.396337 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 18:42:22.396546 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 18:42:22.396976 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":98,"spinlock_wait_cycles":25472,"thread_start_us":154,"threads_started":1}
I20260806 18:42:22.397855 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.377622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49742 (local address 127.8.241.254:42957)
0806 18:42:22.378067 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.378088 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.378350 (+ 262us) server_negotiation.cc:407] Connection header received
0806 18:42:22.379401 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.379406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.379619 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.379795 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.380715 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.381660 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.382664 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.383139 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.383282 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.386637 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.386644 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.386655 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.386679 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.394567 (+ 7888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.395110 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.395117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.395119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.395280 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.395567 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.395570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.395571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.396042 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.396157 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.396622 (+ 465us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.396905 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":296,"thread_start_us":130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab.
Entry for principal kudu/127.8.241.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab.
I20260806 18:42:22.400377 29675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:42:22.401223 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.400898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:44311 (local address 127.0.0.1:38770)
0806 18:42:22.401094 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.401163 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:42:22.401345 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.253:44311
--webserver_interface=127.8.241.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab
--principal=kudu/127.8.241.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:42957,127.8.241.253:44311,127.8.241.252:36847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:22.401454 29604 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
I20260806 18:42:22.402310 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.402084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:44311 (local address 127.0.0.1:38784)
0806 18:42:22.402193 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.402248 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:42:22.402452 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.253:44311: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
I20260806 18:42:22.451704 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } attempt: 1
I20260806 18:42:22.452564 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.452267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:44311 (local address 127.0.0.1:38798)
0806 18:42:22.452416 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.452482 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 18:42:22.453387 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.453156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:44311 (local address 127.0.0.1:38802)
0806 18:42:22.453262 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.453313 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 18:42:22.454308 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.253:44311: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
I20260806 18:42:22.522506 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } attempt: 2
I20260806 18:42:22.523372 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.523078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:44311 (local address 127.0.0.1:38806)
0806 18:42:22.523225 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.523288 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:42:22.524144 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.523937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:44311 (local address 127.0.0.1:38810)
0806 18:42:22.524033 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.524082 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 18:42:22.524283 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.253:44311: Network error: Client connection negotiation failed: client connection to 127.8.241.253:44311: connect: Connection refused (error 111)
W20260806 18:42:22.530398 29684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:22.530646 29684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:22.530689 29684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:22.534792 29684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:22.534852 29684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:22.534874 29684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:22.534899 29684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:22.534917 29684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:22.540234 29684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:42957,127.8.241.253:44311,127.8.241.252:36847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab
--principal=kudu/127.8.241.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.253:44311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.241.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:22.541308 29684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:22.542330 29684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:22.550913 29694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:22.550928 29690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:22.550915 29689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:22.551364 29684 server_base.cc:1034] running on GCE node
I20260806 18:42:22.551949 29684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:22.553102 29684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:22.554270 29684 hybrid_clock.cc:648] HybridClock initialized: now 1786041742554232 us; error 46 us; skew 500 ppm
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:22.556945 29684 init.cc:377] Logged in from keytab as kudu/127.8.241.253@KRBTEST.COM (short username kudu)
I20260806 18:42:22.557869 29684 webserver.cc:460] Webserver started at http://127.8.241.253:44181/ using document root <none> and password file <none>
I20260806 18:42:22.558439 29684 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:22.558506 29684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:22.558712 29684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:22.561277 29684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/data/instance:
uuid: "040ec8180f7f4ac38983f65bb991568b"
format_stamp: "Formatted at 2026-08-06 18:42:22 on dist-test-slave-k5rr"
server_key: "ff807448195ad6970bd45cfcfb541b6b"
server_key_iv: "2f8e88faea19b084a1ca445fb1a2845a"
server_key_version: "encryptionkey@0"
I20260806 18:42:22.561866 29684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/wal/instance:
uuid: "040ec8180f7f4ac38983f65bb991568b"
format_stamp: "Formatted at 2026-08-06 18:42:22 on dist-test-slave-k5rr"
server_key: "ff807448195ad6970bd45cfcfb541b6b"
server_key_iv: "2f8e88faea19b084a1ca445fb1a2845a"
server_key_version: "encryptionkey@0"
I20260806 18:42:22.565790 29684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:42:22.568234 29708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:22.569475 29684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:42:22.569595 29684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/wal
uuid: "040ec8180f7f4ac38983f65bb991568b"
format_stamp: "Formatted at 2026-08-06 18:42:22 on dist-test-slave-k5rr"
server_key: "ff807448195ad6970bd45cfcfb541b6b"
server_key_iv: "2f8e88faea19b084a1ca445fb1a2845a"
server_key_version: "encryptionkey@0"
I20260806 18:42:22.569694 29684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:22.592103 29684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:22.595124 29684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:22.595327 29684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:22.602600 29684 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.253:44311
I20260806 18:42:22.602653 29783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.253:44311 every 8 connection(s)
I20260806 18:42:22.603619 29684 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
I20260806 18:42:22.606480 29785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:22.608584 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 29684
I20260806 18:42:22.608711 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-1/wal/instance
I20260806 18:42:22.608981 9159 external_mini_cluster.cc:1428] Setting key d5aa5e623370fcbd21fe76d6d17e3141
I20260806 18:42:22.611151 29785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:42:22.615430 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } attempt: 3
WARNING: no policy specified for kudu/127.8.241.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.252@KRBTEST.COM" created.
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:22.626152 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.615809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:44311 (local address 127.0.0.1:38816)
0806 18:42:22.615923 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.615946 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:22.616028 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:22.616725 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:22.616728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:22.616902 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:22.617037 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.617045 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.618456 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.618460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.618969 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.618977 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.619607 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.619611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.619720 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.620215 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:22.620230 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:22.621962 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:22.624233 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.624236 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.624239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.624468 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.624829 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.624833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.624835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.624911 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.625641 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:22.625643 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:22.625735 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:42:22.625829 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:42:22.625984 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:42:22.626341 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.613160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49744 (local address 127.8.241.254:42957)
0806 18:42:22.613326 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.613331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.614252 (+ 921us) server_negotiation.cc:407] Connection header received
0806 18:42:22.615455 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.615458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.615519 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.615595 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.617211 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.618009 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.619133 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.619848 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.619948 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.622723 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.622727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.622728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.622750 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.624294 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.624781 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.624784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.624786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.624871 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.625248 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.625252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.625253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.625460 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.625560 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.626059 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.626190 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:42:22.627494 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.613527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42957 (local address 127.0.0.1:49744)
0806 18:42:22.614063 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.614128 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:22.615218 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:22.615797 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:22.615806 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:22.616280 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:22.616992 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.617039 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.618141 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.618145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.618977 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.618990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.620000 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.620004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.620081 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.620593 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:22.620613 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:22.622590 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:22.624406 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.624415 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.624428 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.624683 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.624970 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.624972 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.624974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.625136 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.625574 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:22.625582 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:22.625769 (+ 187us) client_negotiation.cc:769] Sending connection context
0806 18:42:22.626493 (+ 724us) client_negotiation.cc:240] Negotiation successful
0806 18:42:22.626752 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":24,"thread_start_us":116,"threads_started":1}
I20260806 18:42:22.627492 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.616073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38816 (local address 127.8.241.253:44311)
0806 18:42:22.616351 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.616362 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.616380 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:42:22.616415 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.616419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.616573 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.616714 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.617174 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.618354 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.619079 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.619484 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.619633 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.622167 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.622174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.622190 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.622217 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.624122 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.624591 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.624598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.624600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.624744 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.625039 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.625043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.625045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.625558 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.625679 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.625944 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.626664 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":94,"threads_started":1}
I20260806 18:42:22.629746 29675 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:42:22.629763 29785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:42:22.630509 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.630198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37066)
0806 18:42:22.630352 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.630429 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 18:42:22.631270 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.631015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37070)
0806 18:42:22.631173 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.631216 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260806 18:42:22.631429 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:36847: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
I20260806 18:42:22.637920 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.630468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:44311 (local address 127.0.0.1:38830)
0806 18:42:22.630659 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.630672 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:22.630763 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:22.630999 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:22.631002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:22.631171 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:22.631284 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.631290 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.632111 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.632115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.632789 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.632800 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.633295 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.633297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.633405 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.633792 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:22.633805 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:22.634992 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:22.636528 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.636531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.636532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.636711 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.636965 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.636968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.636969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.637086 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.637437 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:22.637440 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:22.637528 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 18:42:22.637609 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:42:22.637733 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":15}
I20260806 18:42:22.637936 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.630501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38830 (local address 127.8.241.253:44311)
0806 18:42:22.630623 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.630628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.630727 (+ 99us) server_negotiation.cc:407] Connection header received
0806 18:42:22.630842 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.630845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.630891 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.630978 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.631399 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.632010 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.632895 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.633208 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.633289 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.635115 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.635118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.635120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.635141 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.636422 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.636785 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.636787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.636788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.636880 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.637155 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.637158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.637159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.637348 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.637456 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.637653 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.637796 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":45}
I20260806 18:42:22.638731 29785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab.
Entry for principal kudu/127.8.241.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab.
I20260806 18:42:22.639413 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.639159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37074)
0806 18:42:22.639274 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.639359 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 18:42:22.639838 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.252:36847
--webserver_interface=127.8.241.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab
--principal=kudu/127.8.241.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:42957,127.8.241.253:44311,127.8.241.252:36847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:22.639990 29715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
I20260806 18:42:22.640829 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.640639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37076)
0806 18:42:22.640737 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.640782 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:42:22.640990 29785 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:36847: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
I20260806 18:42:22.662603 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } attempt: 1
I20260806 18:42:22.663489 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.663119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37092)
0806 18:42:22.663296 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.663375 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 18:42:22.664248 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.664020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37108)
0806 18:42:22.664123 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.664178 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 18:42:22.664386 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:36847: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
I20260806 18:42:22.690346 29785 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } attempt: 1
I20260806 18:42:22.691255 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.690949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37120)
0806 18:42:22.691109 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.691171 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 18:42:22.691956 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.691734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37124)
0806 18:42:22.691842 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.691888 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 18:42:22.692075 29785 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:36847: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
I20260806 18:42:22.739575 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } attempt: 2
I20260806 18:42:22.740463 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.740091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37136)
0806 18:42:22.740270 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.740351 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 18:42:22.741210 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.740972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37138)
0806 18:42:22.741091 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.741146 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 18:42:22.741340 29675 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:36847: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
W20260806 18:42:22.755245 29796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:22.755486 29796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:22.755533 29796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:22.759477 29796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:22.759531 29796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:22.759548 29796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:22.759578 29796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:22.759604 29796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:22.760211 29785 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } attempt: 2
I20260806 18:42:22.760944 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.760660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37150)
0806 18:42:22.760797 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.760855 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 18:42:22.761662 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.761483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37162)
0806 18:42:22.761567 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.761610 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 18:42:22.761776 29785 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:36847: Network error: Client connection negotiation failed: client connection to 127.8.241.252:36847: connect: Connection refused (error 111)
I20260806 18:42:22.764990 29796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:42957,127.8.241.253:44311,127.8.241.252:36847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab
--principal=kudu/127.8.241.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.252:36847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.241.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:22.766053 29796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:22.767043 29796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:22.775524 29806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:22.775575 29807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:22.775650 29796 server_base.cc:1034] running on GCE node
W20260806 18:42:22.775524 29810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:22.776268 29796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:22.777348 29796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:22.778509 29796 hybrid_clock.cc:648] HybridClock initialized: now 1786041742778491 us; error 30 us; skew 500 ppm
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:22.781153 29796 init.cc:377] Logged in from keytab as kudu/127.8.241.252@KRBTEST.COM (short username kudu)
I20260806 18:42:22.782006 29796 webserver.cc:460] Webserver started at http://127.8.241.252:38473/ using document root <none> and password file <none>
I20260806 18:42:22.782536 29796 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:22.782595 29796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:22.782793 29796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:22.784858 29796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/data/instance:
uuid: "c2ba369dcab64b85b237b0f104045f2e"
format_stamp: "Formatted at 2026-08-06 18:42:22 on dist-test-slave-k5rr"
server_key: "45780dee3dfe76f5ddf45462b0da338d"
server_key_iv: "c77d94c4d879eb2491be25ca1459952d"
server_key_version: "encryptionkey@0"
I20260806 18:42:22.785456 29796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/wal/instance:
uuid: "c2ba369dcab64b85b237b0f104045f2e"
format_stamp: "Formatted at 2026-08-06 18:42:22 on dist-test-slave-k5rr"
server_key: "45780dee3dfe76f5ddf45462b0da338d"
server_key_iv: "c77d94c4d879eb2491be25ca1459952d"
server_key_version: "encryptionkey@0"
I20260806 18:42:22.789113 29796 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:42:22.791519 29818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:22.792707 29796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:42:22.792817 29796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/wal
uuid: "c2ba369dcab64b85b237b0f104045f2e"
format_stamp: "Formatted at 2026-08-06 18:42:22 on dist-test-slave-k5rr"
server_key: "45780dee3dfe76f5ddf45462b0da338d"
server_key_iv: "c77d94c4d879eb2491be25ca1459952d"
server_key_version: "encryptionkey@0"
I20260806 18:42:22.792925 29796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:22.820858 29796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:22.824193 29796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:22.824384 29796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:22.831504 29796 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.252:36847
I20260806 18:42:22.831519 29891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.252:36847 every 8 connection(s)
I20260806 18:42:22.832479 29796 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
I20260806 18:42:22.835371 29892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:22.836489 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 29796
I20260806 18:42:22.836604 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041577757111-9159-0/minicluster-data/master-2/wal/instance
I20260806 18:42:22.836855 9159 external_mini_cluster.cc:1428] Setting key 6f5227c417d45cdff7de7e489af019a7
I20260806 18:42:22.839639 29892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:42:22.840493 29675 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } attempt: 3
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
I20260806 18:42:22.852012 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.841355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49748 (local address 127.8.241.254:42957)
0806 18:42:22.841474 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.841478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.841941 (+ 463us) server_negotiation.cc:407] Connection header received
0806 18:42:22.842122 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.842125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.842170 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.842248 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.843137 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.843888 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.844991 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.845310 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.845408 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.848077 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.848080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.848082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.848103 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.849591 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.850226 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.850230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.850232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.850315 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.850697 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.850701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.850703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.851023 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.851116 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.851688 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.851840 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:42:22.852123 29681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.840914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37176)
0806 18:42:22.841112 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.841127 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:22.841215 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:22.841503 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:22.841506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:22.841686 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:22.841791 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.841798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.844013 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.844015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.844611 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.844622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.845580 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.845583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.845659 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.846029 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:22.846043 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:22.847355 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:22.850441 (+ 3086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.850444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.850446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.850657 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.850918 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.850921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.850923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.851003 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.851530 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:22.851533 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:22.851627 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 18:42:22.851726 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:42:22.851928 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":46,"spinlock_wait_cycles":768}
I20260806 18:42:22.852916 29785 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } attempt: 3
I20260806 18:42:22.853082 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.841324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42957 (local address 127.0.0.1:49748)
0806 18:42:22.841829 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.841873 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:22.842023 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:22.842373 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:22.842379 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:22.842781 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:22.842970 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.842980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.843986 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.843989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.844896 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.844905 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.845454 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.845460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.845566 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.846168 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:22.846184 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:22.847925 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:22.849818 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.849826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.849834 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.850135 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.850398 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.850401 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.850404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.850585 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.851108 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:22.851114 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:22.851411 (+ 297us) client_negotiation.cc:769] Sending connection context
0806 18:42:22.852590 (+ 1179us) client_negotiation.cc:240] Negotiation successful
0806 18:42:22.852777 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":22,"spinlock_wait_cycles":1408,"thread_start_us":141,"threads_started":1}
I20260806 18:42:22.853219 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.838266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.8.241.252:36847)
0806 18:42:22.838681 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.838697 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.838733 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:42:22.839409 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.839427 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.839758 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.840059 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.840883 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.842982 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.844123 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.844823 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.844980 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.848609 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.848613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.848615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.848642 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.850459 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.850881 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.850884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.850886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.850966 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.851275 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.851278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.851280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.851479 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.851641 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.852393 (+ 752us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.852507 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":267,"thread_start_us":121,"threads_started":1}
I20260806 18:42:22.853097 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.840959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.8.241.252:36847)
0806 18:42:22.841240 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.841249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.841263 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:42:22.841299 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.841302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.841340 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.841462 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.841933 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.842986 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.844719 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.845483 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.845578 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.847503 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.847511 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.847524 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.847552 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.850314 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.850747 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.850753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.850755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.850831 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.851104 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.851108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.851109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.851415 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.851547 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.851840 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.852137 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":703,"server-negotiator.queue_time_us":220,"thread_start_us":113,"threads_started":1}
I20260806 18:42:22.854640 29892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
I20260806 18:42:22.859762 29675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38: Bootstrap starting.
I20260806 18:42:22.862474 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.853503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37192)
0806 18:42:22.853646 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.853661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:22.853744 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:22.854021 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:22.854024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:22.854208 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:22.854322 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.854330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.855166 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.855170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.855861 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.855879 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.856465 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.856469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.856546 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.856987 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:22.857001 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:22.858421 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:22.860744 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.860747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.860749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.861046 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.861363 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.861367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.861369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.861460 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.861931 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:22.861934 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:22.862031 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 18:42:22.862184 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 18:42:22.862311 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:42:22.862874 29675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38: Neither blocks nor log segments found. Creating new log.
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:22.863118 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.853585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37192 (local address 127.8.241.252:36847)
0806 18:42:22.853741 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.853745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.853758 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:42:22.853862 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.853866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.853906 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.854426 (+ 520us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.854494 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.855046 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.856011 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.856332 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.856772 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.858568 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.858572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.858575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.858597 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.860618 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.861176 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.861180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.861182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.861257 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.861577 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.861580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.861582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.861806 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.862774 (+ 968us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.862843 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.862970 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":96,"threads_started":1}
I20260806 18:42:22.863950 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.855169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38846 (local address 127.8.241.253:44311)
0806 18:42:22.855283 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.855287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.855299 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:42:22.855388 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.855391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.855433 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.855508 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.855941 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.856428 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.857332 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.857669 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.857745 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.859989 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.859993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.859995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.860030 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.861821 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.862239 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.862242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.862244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.862316 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.862882 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.862886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.862887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.863085 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.863156 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.863720 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.863830 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:42:22.864300 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.855077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:44311 (local address 127.0.0.1:38846)
0806 18:42:22.855228 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.855240 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:22.855308 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:22.855540 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:22.855543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:22.855697 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:22.855807 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.855814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.856533 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.856536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.857200 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.857212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.857900 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.857904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.858006 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.858421 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:22.858435 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:22.859867 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:22.861923 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.861926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.861927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.862140 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.862462 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.862465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.862466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.862780 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.863450 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:22.863452 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:22.863545 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 18:42:22.864077 (+ 532us) client_negotiation.cc:240] Negotiation successful
0806 18:42:22.864203 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 18:42:22.865437 29675 log.cc:824] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:22.865648 29892 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:42:22.866230 29785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b: Bootstrap starting.
I20260806 18:42:22.867364 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.857635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.8.241.254:42957)
0806 18:42:22.857736 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.857741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.858030 (+ 289us) server_negotiation.cc:407] Connection header received
0806 18:42:22.858069 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.858072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.858115 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.858182 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.858982 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.859645 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.860644 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.860992 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.861127 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.863654 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.863658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.863661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.863685 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.865288 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.865761 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.865764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.865766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.865851 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.866252 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.866256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.866257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.866469 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.866600 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.867114 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.867225 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:42:22.868924 29785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:22.869834 29785 log.cc:824] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
I20260806 18:42:22.871995 29675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38: No bootstrap required, opened a new log
I20260806 18:42:22.872650 29785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b: No bootstrap required, opened a new log
Aug 06 18:42:22 dist-test-slave-k5rr krb5kdc[29557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
I20260806 18:42:22.874830 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.866185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.8.241.252:36847)
0806 18:42:22.866677 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.866680 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.866690 (+ 10us) server_negotiation.cc:407] Connection header received
0806 18:42:22.866716 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.866718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.866748 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.866810 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.867156 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.867713 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.868689 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.868972 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.869120 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.871692 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.871696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.871698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.871721 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.873221 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.873713 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.873717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.873718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.873796 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.874049 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.874051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.874053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.874225 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.874286 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.874584 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.874690 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432}
I20260806 18:42:22.875269 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.866151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:36847 (local address 127.0.0.1:37198)
0806 18:42:22.866258 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:22.866272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:22.866341 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:22.866843 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:22.866846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:22.866987 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:22.867070 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.867076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.867807 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.867810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.868576 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:22.868587 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.869125 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.869129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:22.869203 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.870183 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:22.870197 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:22.871560 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:22.873328 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.873331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.873333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.873548 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.873895 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:22.873897 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:22.873899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:22.873971 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:22.874340 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:22.874343 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:22.874421 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 18:42:22.875011 (+ 590us) client_negotiation.cc:240] Negotiation successful
0806 18:42:22.875136 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:42:22.876281 29675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } }
I20260806 18:42:22.876531 29675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:22.876600 29675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94200570712046f49d5c9e4a10cbcc38, State: Initialized, Role: FOLLOWER
I20260806 18:42:22.877216 29675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } }
I20260806 18:42:22.877477 29785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } }
I20260806 18:42:22.877696 29785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:22.877753 29785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040ec8180f7f4ac38983f65bb991568b, State: Initialized, Role: FOLLOWER
I20260806 18:42:22.878180 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.868462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38850 (local address 127.8.241.253:44311)
0806 18:42:22.868553 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.868557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.868621 (+ 64us) server_negotiation.cc:407] Connection header received
0806 18:42:22.868713 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.868716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.868757 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.868822 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.869551 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.870057 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.871015 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.871303 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.871376 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.874071 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.874074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.874077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.874101 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.875677 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.876355 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.876359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.876360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.876445 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.876938 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.876941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.876943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.877179 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.877337 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.877928 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.878044 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 18:42:22.878216 29785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } }
I20260806 18:42:22.878386 29892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e: Bootstrap starting.
I20260806 18:42:22.878734 29675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:22.879465 29785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:22.881253 29892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:22.882192 29892 log.cc:824] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:22.882385 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:42:22.885463 29905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.885697 29905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [sys.catalog]: This master's current role is: FOLLOWER
W20260806 18:42:22.888298 29930 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:42:22.888373 29930 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 18:42:22.888820 29939 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:42:22.888890 29939 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:42:22.888841 29903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.888957 29903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:42:22.886968 29576 master_runner.cc:186] Error getting master registration for 127.8.241.252:36847: OK, instance_id { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" instance_seqno: 1786041742829560 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:42:22.890213 29576 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:42:22.890482 29903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:22.890614 29903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } }
I20260806 18:42:22.891918 29751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "94200570712046f49d5c9e4a10cbcc38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040ec8180f7f4ac38983f65bb991568b" is_pre_election: true
I20260806 18:42:22.892191 29751 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94200570712046f49d5c9e4a10cbcc38 in term 0.
I20260806 18:42:22.892601 29604 leader_election.cc:302] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 040ec8180f7f4ac38983f65bb991568b, 94200570712046f49d5c9e4a10cbcc38; no voters:
I20260806 18:42:22.893091 29903 leader_election.cc:288] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 040ec8180f7f4ac38983f65bb991568b (127.8.241.253:44311), c2ba369dcab64b85b237b0f104045f2e (127.8.241.252:36847)
I20260806 18:42:22.893200 29903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:42:22.893285 29903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:42:22.893329 29903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:22.894021 29679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.885135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49774 (local address 127.8.241.254:42957)
0806 18:42:22.885259 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.885263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.885277 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:42:22.885317 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.885320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.885364 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.885449 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.886334 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.887068 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.888198 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.888498 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.888573 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.890400 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.890404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.890407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.890432 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.892059 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.892552 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.892556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.892557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.892646 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.892984 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.892988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.892989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.893245 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.893378 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.893754 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.893866 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:42:22.894533 29903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } }
I20260806 18:42:22.894884 29903 leader_election.cc:288] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [CANDIDATE]: Term 1 election: Requested vote from peers 040ec8180f7f4ac38983f65bb991568b (127.8.241.253:44311), c2ba369dcab64b85b237b0f104045f2e (127.8.241.252:36847)
I20260806 18:42:22.895202 29892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e: No bootstrap required, opened a new log
I20260806 18:42:22.890460 29684 master_runner.cc:186] Error getting master registration for 127.8.241.252:36847: OK, instance_id { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" instance_seqno: 1786041742829560 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:42:22.895471 29684 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:42:22.895474 29751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "94200570712046f49d5c9e4a10cbcc38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040ec8180f7f4ac38983f65bb991568b"
I20260806 18:42:22.895596 29751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:22.896870 29751 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94200570712046f49d5c9e4a10cbcc38 in term 1.
I20260806 18:42:22.897785 29604 leader_election.cc:302] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 040ec8180f7f4ac38983f65bb991568b, 94200570712046f49d5c9e4a10cbcc38; no voters:
I20260806 18:42:22.898989 29903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:42:22.899080 29903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [term 1 LEADER]: Becoming Leader. State: Replica: 94200570712046f49d5c9e4a10cbcc38, State: Running, Role: LEADER
I20260806 18:42:22.899372 29903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } }
I20260806 18:42:22.899418 29892 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } }
I20260806 18:42:22.899668 29892 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:22.899725 29892 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ba369dcab64b85b237b0f104045f2e, State: Initialized, Role: FOLLOWER
I20260806 18:42:22.900447 29892 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } }
I20260806 18:42:22.901199 29954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94200570712046f49d5c9e4a10cbcc38. Latest consensus state: current_term: 1 leader_uuid: "94200570712046f49d5c9e4a10cbcc38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.901320 29954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:22.901165 29955 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.901558 29955 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:42:22.903062 29856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "94200570712046f49d5c9e4a10cbcc38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ba369dcab64b85b237b0f104045f2e"
I20260806 18:42:22.903331 29856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:22.904707 29856 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94200570712046f49d5c9e4a10cbcc38 in term 1.
I20260806 18:42:22.903090 29857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "94200570712046f49d5c9e4a10cbcc38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2ba369dcab64b85b237b0f104045f2e" is_pre_election: true
I20260806 18:42:22.905045 29857 raft_consensus.cc:2362] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 94200570712046f49d5c9e4a10cbcc38 in term 1. Re-sending same reply.
I20260806 18:42:22.902664 29892 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:22.907068 29963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:42:22.907191 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.885583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38866 (local address 127.8.241.253:44311)
0806 18:42:22.885742 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.885746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.895658 (+ 9912us) server_negotiation.cc:407] Connection header received
0806 18:42:22.895825 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.895828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.895874 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.895977 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.896813 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.897604 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.898966 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.899305 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.899411 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.902704 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.902709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.902712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.902739 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.904533 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.905264 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.905268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.905270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.905362 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.905728 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.905732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.905734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.905980 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.906116 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.906904 (+ 788us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.907034 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260806 18:42:22.910355 29976 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 18:42:22.911409 29963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 18:42:22.910444 29976 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:42:22.917438 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:22.889452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37202 (local address 127.8.241.252:36847)
0806 18:42:22.889563 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:22.889567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:22.906795 (+ 17228us) server_negotiation.cc:407] Connection header received
0806 18:42:22.908494 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:22.908497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:22.908548 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:22.908597 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:22.909442 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.910127 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.911171 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:22.911520 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:22.911599 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:22.913531 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:22.913535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:22.913537 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:22.913563 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:22.915164 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.915662 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.915665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.915667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.915755 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:22.916050 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:22.916053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:22.916055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:22.916287 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:22.916409 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:22.917143 (+ 734us) server_negotiation.cc:299] Negotiation successful
0806 18:42:22.917281 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:42:22.925000 29857 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [term 1 FOLLOWER]: Refusing update from remote peer 94200570712046f49d5c9e4a10cbcc38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:22.925738 29954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:22.925967 29751 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [term 1 FOLLOWER]: Refusing update from remote peer 94200570712046f49d5c9e4a10cbcc38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:22.926548 29954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:22.929955 29955 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94200570712046f49d5c9e4a10cbcc38. Latest consensus state: current_term: 1 leader_uuid: "94200570712046f49d5c9e4a10cbcc38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.930086 29955 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:42:22.930823 29905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94200570712046f49d5c9e4a10cbcc38. Latest consensus state: current_term: 1 leader_uuid: "94200570712046f49d5c9e4a10cbcc38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.930965 29905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:42:22.933666 29954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94200570712046f49d5c9e4a10cbcc38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.933787 29954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:22.935168 29905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94200570712046f49d5c9e4a10cbcc38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.935279 29905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:42:22.935583 29978 mvcc.cc:205] Tried to move back new op lower bound from 7315626979012186112 to 7315626978919038976. Current Snapshot: MvccSnapshot[applied={T|T < 7315626979012186112}]
I20260806 18:42:22.938047 29980 mvcc.cc:205] Tried to move back new op lower bound from 7315626979012186112 to 7315626978919038976. Current Snapshot: MvccSnapshot[applied={T|T < 7315626979012186112}]
I20260806 18:42:22.940196 29903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94200570712046f49d5c9e4a10cbcc38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.940338 29903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:22.941074 29955 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94200570712046f49d5c9e4a10cbcc38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94200570712046f49d5c9e4a10cbcc38" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42957 } } peers { permanent_uuid: "040ec8180f7f4ac38983f65bb991568b" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 44311 } } peers { permanent_uuid: "c2ba369dcab64b85b237b0f104045f2e" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 36847 } } }
I20260806 18:42:22.941242 29955 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:42:22.941439 29963 catalog_manager.cc:1347] Generated new cluster ID: 6ca08d339b37410289147b4c5e04efe1
I20260806 18:42:22.941501 29963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:42:22.942827 29979 mvcc.cc:205] Tried to move back new op lower bound from 7315626979012186112 to 7315626978919038976. Current Snapshot: MvccSnapshot[applied={T|T < 7315626979012186112}]
I20260806 18:42:22.969659 29963 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:42:22.971016 29963 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:42:22.993770 29963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94200570712046f49d5c9e4a10cbcc38: Generated new TSK 0
I20260806 18:42:22.994742 29963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:42:23.890252 29939 catalog_manager.cc:1259] Loaded cluster ID: 6ca08d339b37410289147b4c5e04efe1
I20260806 18:42:23.890342 29939 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b: loading cluster ID for follower catalog manager: success
I20260806 18:42:23.892102 29939 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b: acquiring CA information for follower catalog manager: success
I20260806 18:42:23.893338 29939 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 040ec8180f7f4ac38983f65bb991568b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:42:23.913439 29976 catalog_manager.cc:1259] Loaded cluster ID: 6ca08d339b37410289147b4c5e04efe1
I20260806 18:42:23.913506 29976 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e: loading cluster ID for follower catalog manager: success
I20260806 18:42:23.915156 29976 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e: acquiring CA information for follower catalog manager: success
I20260806 18:42:23.916340 29976 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2ba369dcab64b85b237b0f104045f2e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:42:24.069368 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 29576
I20260806 18:42:24.088083 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 29684
I20260806 18:42:24.103149 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 29796
2026-08-06T18:42:24Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3188 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:42:24 dist-test-slave-k5rr krb5kdc[29994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:42:24 dist-test-slave-k5rr krb5kdc[29994](info): set up 2 sockets
Aug 06 18:42:24 dist-test-slave-k5rr krb5kdc[29994](info): commencing operation
krb5kdc: starting...
W20260806 18:42:25.242699 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:42:25 dist-test-slave-k5rr krb5kdc[29994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:42:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:42:25.363653 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45789
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:38783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:45789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:25.476533 30007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:25.476824 30007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:25.476907 30007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:25.482343 30007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:25.482422 30007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:25.482460 30007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:25.482491 30007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:25.482513 30007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:25.488560 30007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:38783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:45789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:45789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:25.489765 30007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:25.490824 30007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:25.499454 30018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:25.499485 30017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:25.499426 30020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:25.499833 30007 server_base.cc:1034] running on GCE node
I20260806 18:42:25.500391 30007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:25.501578 30007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:25.502768 30007 hybrid_clock.cc:648] HybridClock initialized: now 1786041745502737 us; error 40 us; skew 500 ppm
Aug 06 18:42:25 dist-test-slave-k5rr krb5kdc[29994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041745, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:25.505600 30007 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:42:25.506544 30007 webserver.cc:460] Webserver started at http://127.8.241.254:42521/ using document root <none> and password file <none>
I20260806 18:42:25.507113 30007 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:25.507182 30007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:25.507428 30007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:25.509774 30007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8"
format_stamp: "Formatted at 2026-08-06 18:42:25 on dist-test-slave-k5rr"
server_key: "87eeb58e18b2f76047387c8a2dfc096c"
server_key_iv: "cbd79340732136840f2d07f75f010cf7"
server_key_version: "encryptionkey@0"
I20260806 18:42:25.510432 30007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8"
format_stamp: "Formatted at 2026-08-06 18:42:25 on dist-test-slave-k5rr"
server_key: "87eeb58e18b2f76047387c8a2dfc096c"
server_key_iv: "cbd79340732136840f2d07f75f010cf7"
server_key_version: "encryptionkey@0"
I20260806 18:42:25.514426 30007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:42:25.516769 30029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:25.517993 30007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:42:25.518107 30007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8"
format_stamp: "Formatted at 2026-08-06 18:42:25 on dist-test-slave-k5rr"
server_key: "87eeb58e18b2f76047387c8a2dfc096c"
server_key_iv: "cbd79340732136840f2d07f75f010cf7"
server_key_version: "encryptionkey@0"
I20260806 18:42:25.518209 30007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:25.541215 30007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:25.543970 30007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:25.544155 30007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:25.551493 30007 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:45789
I20260806 18:42:25.551525 30105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:45789 every 8 connection(s)
I20260806 18:42:25.552625 30007 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:42:25.555521 30106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:25.560515 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 30007
I20260806 18:42:25.560612 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:42:25.560876 9159 external_mini_cluster.cc:1428] Setting key adc49fa43298dd4a6d1256a007d62346
I20260806 18:42:25.561695 30106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8: Bootstrap starting.
I20260806 18:42:25.564023 30106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:25.564965 30106 log.cc:824] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:25.567086 30106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8: No bootstrap required, opened a new log
Aug 06 18:42:25 dist-test-slave-k5rr krb5kdc[29994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:25.571372 30106 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45789 } }
I20260806 18:42:25.571619 30106 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:25.571669 30106 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec9a651d9ee14e4b8ed6b247434dc7a8, State: Initialized, Role: FOLLOWER
I20260806 18:42:25.572093 30106 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45789 } }
I20260806 18:42:25.572217 30106 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:42:25.572276 30106 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:42:25.572373 30106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:25.573479 30106 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45789 } }
I20260806 18:42:25.573828 30106 leader_election.cc:302] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec9a651d9ee14e4b8ed6b247434dc7a8; no voters:
I20260806 18:42:25.574193 30106 leader_election.cc:288] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:42:25.574342 30114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:42:25.574582 30114 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [term 1 LEADER]: Becoming Leader. State: Replica: ec9a651d9ee14e4b8ed6b247434dc7a8, State: Running, Role: LEADER
I20260806 18:42:25.574958 30114 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45789 } }
I20260806 18:42:25.575523 30106 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:25.575668 30111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:25.562204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52678 (local address 127.8.241.254:45789)
0806 18:42:25.562626 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:25.562641 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:25.562677 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:42:25.563304 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:25.563321 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:25.563639 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:25.563929 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:25.564859 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:25.566088 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:25.567233 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:25.567724 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:25.567850 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:25.570618 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:25.570624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:25.570637 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:25.570665 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:25.572379 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:25.572833 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:25.572838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:25.572842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:25.572963 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:25.573269 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:25.573271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:25.573272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:25.573715 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:25.573870 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:25.574171 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 18:42:25.574571 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260806 18:42:25.577643 30116 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec9a651d9ee14e4b8ed6b247434dc7a8. Latest consensus state: current_term: 1 leader_uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45789 } } }
I20260806 18:42:25.578240 30116 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:25.577859 30115 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec9a651d9ee14e4b8ed6b247434dc7a8" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 45789 } } }
I20260806 18:42:25.578519 30115 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:25.578953 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 18:42:25.580602 30133 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:42:25.580682 30133 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:42:25.580978 30136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:42:25.581918 30136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:42:25.587476 30136 catalog_manager.cc:1347] Generated new cluster ID: 057390e3e7f647818ddd7c5cd1d0a87b
I20260806 18:42:25.587551 30136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:42:25.607321 30136 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:42:25.608333 30136 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:42:25.615355 30136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec9a651d9ee14e4b8ed6b247434dc7a8: Generated new TSK 0
I20260806 18:42:25.616082 30136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:42:25.635092 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 30007
2026-08-06T18:42:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1534 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:42:25 dist-test-slave-k5rr krb5kdc[30144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:42:25 dist-test-slave-k5rr krb5kdc[30144](info): set up 2 sockets
Aug 06 18:42:25 dist-test-slave-k5rr krb5kdc[30144](info): commencing operation
krb5kdc: starting...
W20260806 18:42:26.774322 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:42:26 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:42:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:42:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:42:26.898097 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42029
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:27.016661 30162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:27.016939 30162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:27.016999 30162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:27.020941 30162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:27.020999 30162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:27.021046 30162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:27.021070 30162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:27.021098 30162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:27.026422 30162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:27.027456 30162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:27.028461 30162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:27.037642 30169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:27.037678 30167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:27.037683 30171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:27.037946 30162 server_base.cc:1034] running on GCE node
I20260806 18:42:27.038543 30162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:27.039659 30162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:27.040844 30162 hybrid_clock.cc:648] HybridClock initialized: now 1786041747040823 us; error 31 us; skew 500 ppm
Aug 06 18:42:27 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041747, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:27.043591 30162 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:42:27.044543 30162 webserver.cc:460] Webserver started at http://127.8.241.254:33241/ using document root <none> and password file <none>
I20260806 18:42:27.045137 30162 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:27.045203 30162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:27.045414 30162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:27.047436 30162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "0ca7f856b4d840799a89192870b79d0a"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "98ce906678c789243f2e4d984e5576ee"
server_key_iv: "d09aa25483481852d877ae4a0b471a66"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.047962 30162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "0ca7f856b4d840799a89192870b79d0a"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "98ce906678c789243f2e4d984e5576ee"
server_key_iv: "d09aa25483481852d877ae4a0b471a66"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.051577 30162 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 18:42:27.053889 30182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.055095 30162 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:42:27.055217 30162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "0ca7f856b4d840799a89192870b79d0a"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "98ce906678c789243f2e4d984e5576ee"
server_key_iv: "d09aa25483481852d877ae4a0b471a66"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.055320 30162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:27.108444 30162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:27.111683 30162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:27.111894 30162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:27.119831 30162 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:42029
I20260806 18:42:27.120107 30255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:42029 every 8 connection(s)
I20260806 18:42:27.120893 30162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:42:27.123879 30259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:27.125324 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 30162
I20260806 18:42:27.125442 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:42:27.125739 9159 external_mini_cluster.cc:1428] Setting key b2e4ba4c52eda30e150467b2647f5cc4
I20260806 18:42:27.130613 30259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Bootstrap starting.
I20260806 18:42:27.133383 30259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:27.134161 30259 log.cc:824] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:42:27 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:27.136313 30259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: No bootstrap required, opened a new log
I20260806 18:42:27.141158 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.127187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36860 (local address 127.8.241.254:42029)
0806 18:42:27.127648 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:27.127670 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:27.127714 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:42:27.128581 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:27.128608 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:27.129085 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:27.129473 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:27.130191 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.131688 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.132582 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.133304 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.133472 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.136017 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:27.136024 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:27.136041 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:27.136073 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:27.137990 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.138412 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.138418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.138424 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.138579 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.138902 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.138906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.138907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.139435 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:27.139625 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:27.139826 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 18:42:27.140141 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":118,"threads_started":1}
I20260806 18:42:27.141058 30259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:27.141430 30259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:27.141517 30259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca7f856b4d840799a89192870b79d0a, State: Initialized, Role: FOLLOWER
I20260806 18:42:27.142030 30259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:27.142192 30259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:42:27.142259 30259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:42:27.142369 30259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:27.143316 30259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:27.143684 30259 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ca7f856b4d840799a89192870b79d0a; no voters:
I20260806 18:42:27.143988 30259 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:42:27.144070 30266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:42:27.144304 30266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 1 LEADER]: Becoming Leader. State: Replica: 0ca7f856b4d840799a89192870b79d0a, State: Running, Role: LEADER
I20260806 18:42:27.144659 30266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:27.145197 30259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:27.146369 30268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ca7f856b4d840799a89192870b79d0a. Latest consensus state: current_term: 1 leader_uuid: "0ca7f856b4d840799a89192870b79d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } } }
I20260806 18:42:27.146356 30267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ca7f856b4d840799a89192870b79d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } } }
I20260806 18:42:27.146512 30268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:27.146512 30267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:27.146880 30279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:42:27.149887 30279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:42:27.154892 30279 catalog_manager.cc:1347] Generated new cluster ID: 7f54009a3bdc40e5a4d1f279804ad244
I20260806 18:42:27.154963 30279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:42:27.164084 30279 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:42:27.165233 30279 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:42:27.175704 30279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Generated new TSK 0
I20260806 18:42:27.176389 30279 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:42:27.211625 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:42029
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:27.327955 30298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:27.328217 30298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:27.328258 30298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:27.332197 30298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:27.332254 30298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:27.332309 30298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:42:27.337513 30298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:27.338542 30298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:27.339511 30298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:27.349200 30306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:27.349195 30303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:27.349200 30304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:27.349462 30298 server_base.cc:1034] running on GCE node
I20260806 18:42:27.349987 30298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:27.350630 30298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:27.351817 30298 hybrid_clock.cc:648] HybridClock initialized: now 1786041747351795 us; error 31 us; skew 500 ppm
Aug 06 18:42:27 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041747, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:27.354679 30298 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:42:27.355547 30298 webserver.cc:460] Webserver started at http://127.8.241.193:44413/ using document root <none> and password file <none>
I20260806 18:42:27.356112 30298 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:27.356180 30298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:27.356375 30298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:27.358596 30298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "b27241ab24d54472b43c1075f1f6ed2f"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "8206aa95fdd04499f5b470c6585e1841"
server_key_iv: "fdbf2ebb48a61c29edcb6b76d11e2220"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.359166 30298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "b27241ab24d54472b43c1075f1f6ed2f"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "8206aa95fdd04499f5b470c6585e1841"
server_key_iv: "fdbf2ebb48a61c29edcb6b76d11e2220"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.362988 30298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:42:27.365446 30317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.366647 30298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:42:27.366761 30298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "b27241ab24d54472b43c1075f1f6ed2f"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "8206aa95fdd04499f5b470c6585e1841"
server_key_iv: "fdbf2ebb48a61c29edcb6b76d11e2220"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.366855 30298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:27.376822 30298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:27.380071 30298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:27.380254 30298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:27.380846 30298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:27.381727 30298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:27.381778 30298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.381832 30298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:27.381861 30298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.390468 30298 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:37403
I20260806 18:42:27.390519 30492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:37403 every 8 connection(s)
I20260806 18:42:27.391360 30298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:42:27.399127 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 30298
I20260806 18:42:27.399238 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:42:27.399541 9159 external_mini_cluster.cc:1428] Setting key a82c80bfd7fa6eb3df9e5aec7274326b
Aug 06 18:42:27 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041747, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:27.408524 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.393191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:42135 (local address 127.8.241.254:42029)
0806 18:42:27.393319 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:27.393323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:27.394390 (+ 1067us) server_negotiation.cc:407] Connection header received
0806 18:42:27.395301 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:27.395304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:27.395352 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:27.395430 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:27.396593 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.397173 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.398455 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.398674 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.398744 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.404859 (+ 6115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:27.404862 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:27.404864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:27.404902 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:27.406430 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.406961 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.406965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.406967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.407069 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.407370 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.407374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.407376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.407623 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:27.407728 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:27.408215 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 18:42:27.408351 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:42:27.409484 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.393549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:42135)
0806 18:42:27.394259 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:27.394303 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:27.395105 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:27.395578 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:27.395585 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:27.395956 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:27.396473 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.396488 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.397299 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.397303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.398347 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.398360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.398836 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.398839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.399018 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.402897 (+ 3879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:27.402919 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:27.404654 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:27.406567 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:27.406572 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:27.406587 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:27.406851 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:27.407156 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:27.407158 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:27.407159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:27.407287 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:27.407713 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:27.407719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:27.407973 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 18:42:27.408134 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 18:42:27.408418 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:42:27.410816 30493 heartbeater.cc:347] Connected to a master server at 127.8.241.254:42029
I20260806 18:42:27.411085 30493 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:27.411626 30493 heartbeater.cc:511] Master 127.8.241.254:42029 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:42:27.413345 30207 ts_manager.cc:194] Registered new tserver with Master: b27241ab24d54472b43c1075f1f6ed2f (127.8.241.193:37403)
I20260806 18:42:27.414505 30207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:42135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:42:27.429484 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:42029
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:27.545895 30500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:27.546142 30500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:27.546192 30500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:27.550160 30500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:27.550215 30500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:27.550272 30500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:42:27.555477 30500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:27.556702 30500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:27.557770 30500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:27.567170 30509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:27.567166 30508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:27.567227 30511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:27.567438 30500 server_base.cc:1034] running on GCE node
I20260806 18:42:27.567837 30500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:27.568437 30500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:27.569609 30500 hybrid_clock.cc:648] HybridClock initialized: now 1786041747569589 us; error 32 us; skew 500 ppm
Aug 06 18:42:27 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041747, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:27.572278 30500 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:42:27.573174 30500 webserver.cc:460] Webserver started at http://127.8.241.194:36773/ using document root <none> and password file <none>
I20260806 18:42:27.573724 30500 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:27.573789 30500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:27.573983 30500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:27.576015 30500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "3041d19d19af49fa84c77d0fa8430903"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "a91359c1d43fcf6f8b41b3d6d9cae651"
server_key_iv: "e9473b497ce51e35985057ea0c0aced7"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.576565 30500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "3041d19d19af49fa84c77d0fa8430903"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "a91359c1d43fcf6f8b41b3d6d9cae651"
server_key_iv: "e9473b497ce51e35985057ea0c0aced7"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.580235 30500 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:42:27.582671 30522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.583945 30500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:42:27.584059 30500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "3041d19d19af49fa84c77d0fa8430903"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "a91359c1d43fcf6f8b41b3d6d9cae651"
server_key_iv: "e9473b497ce51e35985057ea0c0aced7"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.584153 30500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:27.601346 30500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:27.604022 30500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:27.604194 30500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:27.604776 30500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:27.605654 30500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:27.605705 30500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.605757 30500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:27.605787 30500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.614393 30500 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:36449
I20260806 18:42:27.614439 30689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:36449 every 8 connection(s)
I20260806 18:42:27.615469 30500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:42:27.616402 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 30500
I20260806 18:42:27.616525 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:42:27.616789 9159 external_mini_cluster.cc:1428] Setting key 833973ebfe15e545a16b99fcf3e0cc7b
Aug 06 18:42:27 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041747, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
I20260806 18:42:27.628830 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.617353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:56073 (local address 127.8.241.254:42029)
0806 18:42:27.617508 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:27.617512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:27.619147 (+ 1635us) server_negotiation.cc:407] Connection header received
0806 18:42:27.620007 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:27.620010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:27.620066 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:27.620140 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:27.621360 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.621886 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.623053 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.623278 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.623364 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.625677 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:27.625681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:27.625683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:27.625712 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:27.626976 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.627466 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.627469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.627471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.627549 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.627817 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.627819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.627820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.628006 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:27.628095 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:27.628546 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 18:42:27.628668 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:42:27.629631 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.617800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:56073)
0806 18:42:27.619012 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:27.619052 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:27.619816 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:27.620302 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:27.620310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:27.620670 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:27.621224 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.621241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.622011 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.622014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.622928 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.622949 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.623417 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.623421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.623577 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.624111 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:27.624127 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:27.625524 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:27.627080 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:27.627085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:27.627099 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:27.627372 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:27.627616 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:27.627619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:27.627620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:27.627752 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:27.628101 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:27.628107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:27.628333 (+ 226us) client_negotiation.cc:769] Sending connection context
0806 18:42:27.628488 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 18:42:27.628761 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"thread_start_us":134,"threads_started":1}
I20260806 18:42:27.630762 30690 heartbeater.cc:347] Connected to a master server at 127.8.241.254:42029
I20260806 18:42:27.631034 30690 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:27.631616 30690 heartbeater.cc:511] Master 127.8.241.254:42029 requested a full tablet report, sending...
I20260806 18:42:27.632668 30207 ts_manager.cc:194] Registered new tserver with Master: 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449)
I20260806 18:42:27.633466 30207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:56073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:42:27.644917 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:42029
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:27.759521 30701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:27.759778 30701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:27.759837 30701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:27.764008 30701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:27.764070 30701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:27.764148 30701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:42:27.769337 30701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:27.770372 30701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:27.771420 30701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:27.780651 30711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:27.780651 30708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:27.780807 30701 server_base.cc:1034] running on GCE node
W20260806 18:42:27.780992 30706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:27.781402 30701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:27.782008 30701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:27.783164 30701 hybrid_clock.cc:648] HybridClock initialized: now 1786041747783141 us; error 35 us; skew 500 ppm
Aug 06 18:42:27 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041747, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:27.785763 30701 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:42:27.786576 30701 webserver.cc:460] Webserver started at http://127.8.241.195:37945/ using document root <none> and password file <none>
I20260806 18:42:27.787110 30701 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:27.787171 30701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:27.787443 30701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:27.789520 30701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "3b5c331882ed41b9a8e582e85d938107"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "6625f7e765544b8530efd82b8b7c1712"
server_key_iv: "ce74319db2eeb83136c207d53c22791e"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.790055 30701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "3b5c331882ed41b9a8e582e85d938107"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "6625f7e765544b8530efd82b8b7c1712"
server_key_iv: "ce74319db2eeb83136c207d53c22791e"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.793888 30701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:42:27.796298 30723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.797590 30701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:42:27.797696 30701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "3b5c331882ed41b9a8e582e85d938107"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "6625f7e765544b8530efd82b8b7c1712"
server_key_iv: "ce74319db2eeb83136c207d53c22791e"
server_key_version: "encryptionkey@0"
I20260806 18:42:27.797806 30701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:27.817574 30701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:27.820446 30701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:27.820631 30701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:27.821362 30701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:27.822258 30701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:27.822306 30701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.822351 30701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:27.822381 30701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:27.831290 30701 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:34217
I20260806 18:42:27.831326 30890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:34217 every 8 connection(s)
I20260806 18:42:27.832206 30701 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
Aug 06 18:42:27 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041747, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:27.841518 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 30701
I20260806 18:42:27.841629 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:42:27.841897 9159 external_mini_cluster.cc:1428] Setting key 4c0fddcd4f7e61af1ac5f201a1563d38
I20260806 18:42:27.844944 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.834060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:44321 (local address 127.8.241.254:42029)
0806 18:42:27.834187 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:27.834192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:27.835169 (+ 977us) server_negotiation.cc:407] Connection header received
0806 18:42:27.836032 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:27.836034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:27.836084 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:27.836155 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:27.837346 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.837877 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.838932 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.839136 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.839203 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.841552 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:27.841555 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:27.841557 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:27.841580 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:27.843009 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.843503 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.843506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.843507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.843584 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.843946 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.843949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.843950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.844151 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:27.844226 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:27.844669 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 18:42:27.844789 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:42:27.845853 30893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.834399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:44321)
0806 18:42:27.835037 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:27.835079 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:27.835840 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:27.836304 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:27.836310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:27.836663 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:27.837206 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.837223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.837976 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.837978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.838842 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.838851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.839242 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.839244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.839385 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.839921 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:27.839949 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:27.841383 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:27.843113 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:27.843119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:27.843134 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:27.843416 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:27.843663 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:27.843665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:27.843667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:27.843852 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:27.844237 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:27.844244 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:27.844482 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 18:42:27.844636 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 18:42:27.844929 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":104,"threads_started":1}
I20260806 18:42:27.846956 30891 heartbeater.cc:347] Connected to a master server at 127.8.241.254:42029
I20260806 18:42:27.847182 30891 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:27.847684 30891 heartbeater.cc:511] Master 127.8.241.254:42029 requested a full tablet report, sending...
I20260806 18:42:27.848726 30207 ts_manager.cc:194] Registered new tserver with Master: 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217)
I20260806 18:42:27.849337 30207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:44321
I20260806 18:42:27.855716 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:42:27.864562 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.857645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36862 (local address 127.8.241.254:42029)
0806 18:42:27.857777 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:27.857782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:27.857964 (+ 182us) server_negotiation.cc:407] Connection header received
0806 18:42:27.858091 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:27.858094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:27.858139 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:27.858198 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:27.858909 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.859422 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.860263 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.860492 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.860580 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.861881 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:27.861885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:27.861887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:27.861914 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:27.862961 (+ 1047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.863361 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.863365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.863367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.863449 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:27.863695 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:27.863698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:27.863700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:27.863935 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:27.864065 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:27.864298 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 18:42:27.864411 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:42:27.867398 30207 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:42:27.870472 30207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:42:27.884681 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.878769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:34217 (local address 127.0.0.1:57804)
0806 18:42:27.879235 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:27.879288 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:27.879419 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:27.880249 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:27.880255 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:27.880278 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:27.880407 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.880413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.881844 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.881847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.883047 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.883058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.884114 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.884118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.884227 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.884275 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 18:42:27.884390 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 18:42:27.884523 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260806 18:42:27.884984 30908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.879290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35120 (local address 127.8.241.194:36449)
0806 18:42:27.879773 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:27.879784 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:27.880015 (+ 231us) server_negotiation.cc:407] Connection header received
0806 18:42:27.880239 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:27.880245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:27.880444 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:27.880572 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:27.880995 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.881924 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.883487 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.884281 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.884382 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.884535 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:27.884630 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 18:42:27.884726 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":150,"threads_started":1}
I20260806 18:42:27.885146 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.878883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36449 (local address 127.0.0.1:35120)
0806 18:42:27.879909 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:27.879923 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:27.880085 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:27.880774 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:27.880784 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:27.880798 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:27.880886 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.880891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.882066 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.882069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.883350 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.883361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.884397 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.884401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.884490 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.884513 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:42:27.884599 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:42:27.884669 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":28,"thread_start_us":120,"threads_started":1}
I20260806 18:42:27.885169 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.879979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:37403 (local address 127.0.0.1:36040)
0806 18:42:27.880646 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:27.880659 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:27.880782 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:27.881273 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:27.881276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:27.881296 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:27.881387 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.881392 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.882623 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.882626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.883655 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:27.883663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.884774 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.884778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:27.884861 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.884883 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:42:27.884950 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:42:27.885055 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":105,"threads_started":1}
I20260806 18:42:27.885358 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.878897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57804 (local address 127.8.241.195:34217)
0806 18:42:27.879519 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:27.879532 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:27.879557 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:42:27.879640 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:27.879645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:27.879866 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:27.880047 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:27.880551 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.881714 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.883183 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.883981 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.884097 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.884262 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:27.885124 (+ 862us) server_negotiation.cc:299] Negotiation successful
0806 18:42:27.885228 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":119,"threads_started":1}
I20260806 18:42:27.885437 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:27.880179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36040 (local address 127.8.241.193:37403)
0806 18:42:27.880569 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:27.880586 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:27.880777 (+ 191us) server_negotiation.cc:407] Connection header received
0806 18:42:27.880888 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:27.880893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:27.881128 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:27.881276 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:27.881485 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.882471 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.883763 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:27.884480 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:27.884574 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:27.884709 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:27.885228 (+ 519us) server_negotiation.cc:299] Negotiation successful
0806 18:42:27.885314 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":95,"threads_started":1}
I20260806 18:42:27.888623 30598 tablet_service.cc:1467] Processing CreateTablet for tablet 0d17f658927d4855815d2c250c52cc82 (DEFAULT_TABLE table=test-table [id=4dfc21a0e83d4ae79020089eea236cc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:27.888634 30792 tablet_service.cc:1467] Processing CreateTablet for tablet 0d17f658927d4855815d2c250c52cc82 (DEFAULT_TABLE table=test-table [id=4dfc21a0e83d4ae79020089eea236cc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:27.888633 30391 tablet_service.cc:1467] Processing CreateTablet for tablet 0d17f658927d4855815d2c250c52cc82 (DEFAULT_TABLE table=test-table [id=4dfc21a0e83d4ae79020089eea236cc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:27.889688 30598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d17f658927d4855815d2c250c52cc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:27.889730 30391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d17f658927d4855815d2c250c52cc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:27.889726 30792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d17f658927d4855815d2c250c52cc82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:27.896000 30911 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Bootstrap starting.
I20260806 18:42:27.897264 30914 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: Bootstrap starting.
I20260806 18:42:27.899261 30914 tablet_bootstrap.cc:654] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:27.899700 30911 tablet_bootstrap.cc:654] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:27.900571 30914 log.cc:824] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:27.900570 30911 log.cc:824] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:27.901371 30913 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Bootstrap starting.
I20260806 18:42:27.902894 30911 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: No bootstrap required, opened a new log
I20260806 18:42:27.903120 30911 ts_tablet_manager.cc:1397] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260806 18:42:27.903661 30913 tablet_bootstrap.cc:654] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:27.904943 30913 log.cc:824] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:27.906304 30914 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: No bootstrap required, opened a new log
I20260806 18:42:27.906502 30914 ts_tablet_manager.cc:1397] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260806 18:42:27.907027 30913 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: No bootstrap required, opened a new log
I20260806 18:42:27.907222 30913 ts_tablet_manager.cc:1397] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:42:27.907024 30911 raft_consensus.cc:348] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:27.907312 30911 raft_consensus.cc:374] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:27.907370 30911 raft_consensus.cc:729] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3041d19d19af49fa84c77d0fa8430903, State: Initialized, Role: FOLLOWER
I20260806 18:42:27.907856 30911 consensus_queue.cc:260] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:27.908592 30690 heartbeater.cc:503] Master 127.8.241.254:42029 was elected leader, sending a full tablet report...
I20260806 18:42:27.908898 30911 ts_tablet_manager.cc:1428] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:42:27.909991 30914 raft_consensus.cc:348] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:27.910380 30914 raft_consensus.cc:374] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:27.910460 30914 raft_consensus.cc:729] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5c331882ed41b9a8e582e85d938107, State: Initialized, Role: FOLLOWER
I20260806 18:42:27.910738 30913 raft_consensus.cc:348] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:27.910990 30913 raft_consensus.cc:374] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:27.910899 30914 consensus_queue.cc:260] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:27.911037 30913 raft_consensus.cc:729] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27241ab24d54472b43c1075f1f6ed2f, State: Initialized, Role: FOLLOWER
I20260806 18:42:27.911511 30913 consensus_queue.cc:260] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:27.911636 30891 heartbeater.cc:503] Master 127.8.241.254:42029 was elected leader, sending a full tablet report...
I20260806 18:42:27.911962 30914 ts_tablet_manager.cc:1428] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260806 18:42:27.912427 30913 ts_tablet_manager.cc:1428] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:42:27.912783 30493 heartbeater.cc:503] Master 127.8.241.254:42029 was elected leader, sending a full tablet report...
I20260806 18:42:28.004766 30922 raft_consensus.cc:482] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:28.004979 30922 raft_consensus.cc:504] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:28.006183 30922 leader_election.cc:288] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449), b27241ab24d54472b43c1075f1f6ed2f (127.8.241.193:37403)
I20260806 18:42:28.010910 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.006567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:33875 (local address 127.8.241.193:37403)
0806 18:42:28.006693 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.006699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.006715 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:42:28.006797 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.006800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.006861 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.006947 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:28.007270 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.008058 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.009633 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.010243 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.010370 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.010425 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.010694 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.010768 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:42:28.010910 30908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.006553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:57895 (local address 127.8.241.194:36449)
0806 18:42:28.006712 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.006717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.007064 (+ 347us) server_negotiation.cc:407] Connection header received
0806 18:42:28.007176 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.007178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.007230 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.007307 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:28.007543 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.008224 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.009615 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.010243 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.010370 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.010425 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.010694 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.010767 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:42:28.010962 30893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.006443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:37403 (local address 127.8.241.195:33875)
0806 18:42:28.006568 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:28.006583 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:28.006686 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:28.006990 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:28.006993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:28.007018 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:28.007149 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.007155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.008167 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.008169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.009499 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.009512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.010370 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.010374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.010544 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.010592 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 18:42:28.010672 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:42:28.010792 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":100}
I20260806 18:42:28.010960 30924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.006443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36449 (local address 127.8.241.195:57895)
0806 18:42:28.006983 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:28.006996 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:28.007084 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:28.007321 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:28.007323 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:28.007339 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:28.007434 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.007439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.008313 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.008316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.009525 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.009532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.010370 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.010374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.010540 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.010592 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 18:42:28.010672 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:42:28.010784 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":248,"thread_start_us":133,"threads_started":1}
I20260806 18:42:28.011693 30422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d17f658927d4855815d2c250c52cc82" candidate_uuid: "3b5c331882ed41b9a8e582e85d938107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27241ab24d54472b43c1075f1f6ed2f" is_pre_election: true
I20260806 18:42:28.011811 30624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d17f658927d4855815d2c250c52cc82" candidate_uuid: "3b5c331882ed41b9a8e582e85d938107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3041d19d19af49fa84c77d0fa8430903" is_pre_election: true
I20260806 18:42:28.012038 30422 raft_consensus.cc:2454] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b5c331882ed41b9a8e582e85d938107 in term 0.
I20260806 18:42:28.012104 30624 raft_consensus.cc:2454] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b5c331882ed41b9a8e582e85d938107 in term 0.
I20260806 18:42:28.012554 30725 leader_election.cc:302] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b5c331882ed41b9a8e582e85d938107, b27241ab24d54472b43c1075f1f6ed2f; no voters:
I20260806 18:42:28.012840 30922 raft_consensus.cc:2781] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:42:28.012935 30922 raft_consensus.cc:482] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:42:28.012986 30922 raft_consensus.cc:3037] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:28.014256 30922 raft_consensus.cc:504] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:28.014606 30922 leader_election.cc:288] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [CANDIDATE]: Term 1 election: Requested vote from peers 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449), b27241ab24d54472b43c1075f1f6ed2f (127.8.241.193:37403)
I20260806 18:42:28.015056 30422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d17f658927d4855815d2c250c52cc82" candidate_uuid: "3b5c331882ed41b9a8e582e85d938107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27241ab24d54472b43c1075f1f6ed2f"
I20260806 18:42:28.015056 30624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d17f658927d4855815d2c250c52cc82" candidate_uuid: "3b5c331882ed41b9a8e582e85d938107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3041d19d19af49fa84c77d0fa8430903"
I20260806 18:42:28.015192 30422 raft_consensus.cc:3037] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:28.015192 30624 raft_consensus.cc:3037] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:28.016541 30422 raft_consensus.cc:2454] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b5c331882ed41b9a8e582e85d938107 in term 1.
I20260806 18:42:28.016541 30624 raft_consensus.cc:2454] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b5c331882ed41b9a8e582e85d938107 in term 1.
I20260806 18:42:28.016866 30724 leader_election.cc:302] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3041d19d19af49fa84c77d0fa8430903, 3b5c331882ed41b9a8e582e85d938107, b27241ab24d54472b43c1075f1f6ed2f; no voters:
I20260806 18:42:28.017082 30922 raft_consensus.cc:2781] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:42:28.017364 30922 raft_consensus.cc:686] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 1 LEADER]: Becoming Leader. State: Replica: 3b5c331882ed41b9a8e582e85d938107, State: Running, Role: LEADER
I20260806 18:42:28.017715 30922 consensus_queue.cc:237] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:28.021027 30207 catalog_manager.cc:5495] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195). New cstate: current_term: 1 leader_uuid: "3b5c331882ed41b9a8e582e85d938107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } health_report { overall_health: UNKNOWN } } }
I20260806 18:42:28.040264 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.036010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57820 (local address 127.8.241.195:34217)
0806 18:42:28.036143 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.036147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.036178 (+ 31us) server_negotiation.cc:407] Connection header received
0806 18:42:28.036262 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.036264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.036319 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.036409 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:42:28.036677 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.037373 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.038570 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.038907 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.039041 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.039401 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:42:28.039874 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:42:28.039979 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.040039 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.040096 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:42:28.046643 30624 raft_consensus.cc:1261] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Refusing update from remote peer 3b5c331882ed41b9a8e582e85d938107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:28.046643 30422 raft_consensus.cc:1261] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Refusing update from remote peer 3b5c331882ed41b9a8e582e85d938107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:28.047407 30925 consensus_queue.cc:1035] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:28.047600 30922 consensus_queue.cc:1035] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:28.056102 30931 mvcc.cc:205] Tried to move back new op lower bound from 7315626999990751232 to 7315626999883337728. Current Snapshot: MvccSnapshot[applied={T|T < 7315626999990751232}]
I20260806 18:42:28.068687 30206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:42:28.068890 30206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:42:28.071655 30206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Sending DeleteTablet for 3 replicas of tablet 0d17f658927d4855815d2c250c52cc82
I20260806 18:42:28.072422 30792 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d17f658927d4855815d2c250c52cc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:28 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:57804
I20260806 18:42:28.072649 30391 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d17f658927d4855815d2c250c52cc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:28 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:36040
I20260806 18:42:28.072772 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 30298
I20260806 18:42:28.072988 30598 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d17f658927d4855815d2c250c52cc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:28 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:35120
I20260806 18:42:28.076964 30945 tablet_replica.cc:330] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: stopping tablet replica
I20260806 18:42:28.077519 30945 raft_consensus.cc:2229] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:42:28.078192 30945 raft_consensus.cc:2258] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:42:28.083561 30945 ts_tablet_manager.cc:1905] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:42:28.087121 30945 ts_tablet_manager.cc:1918] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:42:28.087230 30945 log.cc:1196] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/0d17f658927d4855815d2c250c52cc82
I20260806 18:42:28.087555 30945 ts_tablet_manager.cc:1939] T 0d17f658927d4855815d2c250c52cc82 P 3b5c331882ed41b9a8e582e85d938107: Deleting consensus metadata
I20260806 18:42:28.088876 30188 catalog_manager.cc:4841] TS 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217): tablet 0d17f658927d4855815d2c250c52cc82 (table test-table [id=4dfc21a0e83d4ae79020089eea236cc0]) successfully deleted
I20260806 18:42:28.093230 30946 tablet_replica.cc:330] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: stopping tablet replica
I20260806 18:42:28.093603 30946 raft_consensus.cc:2229] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:42:28.093986 30946 raft_consensus.cc:2258] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 18:42:28.094573 30184 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:37403 (error 108)
I20260806 18:42:28.094856 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 30500
I20260806 18:42:28.097599 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.097224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:37403 (local address 127.0.0.1:36054)
0806 18:42:28.097409 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:28.097505 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260806 18:42:28.097779 30184 catalog_manager.cc:4584] TS b27241ab24d54472b43c1075f1f6ed2f (127.8.241.193:37403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0d17f658927d4855815d2c250c52cc82: Network error: Client connection negotiation failed: client connection to 127.8.241.193:37403: connect: Connection refused (error 111)
I20260806 18:42:28.107446 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 30701
I20260806 18:42:28.107651 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.107343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36449 (local address 127.0.0.1:35128)
0806 18:42:28.107490 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:28.107556 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.194:36449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:42:28.120599 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 30162
I20260806 18:42:28.133142 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42029
--webserver_interface=127.8.241.254
--webserver_port=33241
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:28.248312 30948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:28.248613 30948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:28.248677 30948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:28.252820 30948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:28.252897 30948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:28.252924 30948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:28.252944 30948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:28.252965 30948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:28.258481 30948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=33241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:28.259582 30948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:28.260620 30948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:42:28.274988 30948 server_base.cc:1034] running on GCE node
W20260806 18:42:28.274919 30960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:28.274922 30963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:28.274919 30961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:28.275794 30948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:28.276953 30948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:28.278141 30948 hybrid_clock.cc:648] HybridClock initialized: now 1786041748278118 us; error 34 us; skew 500 ppm
Aug 06 18:42:28 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:28.280793 30948 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:42:28.281761 30948 webserver.cc:460] Webserver started at http://127.8.241.254:33241/ using document root <none> and password file <none>
I20260806 18:42:28.282351 30948 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:28.282436 30948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:28.286365 30948 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:42:28.288431 30976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:28.289711 30948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:42:28.289839 30948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "0ca7f856b4d840799a89192870b79d0a"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "98ce906678c789243f2e4d984e5576ee"
server_key_iv: "d09aa25483481852d877ae4a0b471a66"
server_key_version: "encryptionkey@0"
I20260806 18:42:28.290444 30948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:28.312387 30948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:28.315536 30948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:28.315718 30948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:28.322768 30948 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:42029
I20260806 18:42:28.323057 31045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:42029 every 8 connection(s)
I20260806 18:42:28.323927 30948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:42:28.328536 31046 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:42:28.329598 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 30948
I20260806 18:42:28.330010 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:37403
--local_ip_for_outbound_sockets=127.8.241.193
--tserver_master_addrs=127.8.241.254:42029
--webserver_port=44413
--webserver_interface=127.8.241.193
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:42:28.330422 31046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Bootstrap starting.
I20260806 18:42:28.348719 31046 log.cc:824] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:28.358153 31046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:42:28.358536 31046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Bootstrap complete.
I20260806 18:42:28.362774 31046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:28.363204 31046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca7f856b4d840799a89192870b79d0a, State: Initialized, Role: FOLLOWER
I20260806 18:42:28.363729 31046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:28.363865 31046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:42:28.363941 31046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:42:28.364043 31046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:42:28.366117 31046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:28.366495 31046 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ca7f856b4d840799a89192870b79d0a; no voters:
I20260806 18:42:28.366847 31046 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:42:28.367003 31052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:42:28.367270 31052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 2 LEADER]: Becoming Leader. State: Replica: 0ca7f856b4d840799a89192870b79d0a, State: Running, Role: LEADER
I20260806 18:42:28.367587 31052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:28.368281 31046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:28.369130 31054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0ca7f856b4d840799a89192870b79d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } } }
I20260806 18:42:28.369270 31054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:28.369504 31054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ca7f856b4d840799a89192870b79d0a. Latest consensus state: current_term: 2 leader_uuid: "0ca7f856b4d840799a89192870b79d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } } }
I20260806 18:42:28.369589 31054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:28.370381 31066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:42:28.375728 31066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:42:28.376387 31066 catalog_manager.cc:1259] Loaded cluster ID: 7f54009a3bdc40e5a4d1f279804ad244
I20260806 18:42:28.376456 31066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:42:28.378036 31066 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 18:42:28.382398 31066 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 18:42:28.388063 31066 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:42:28.388860 31066 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Loaded TSK: 0
I20260806 18:42:28.390137 31066 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:42:28.453382 31050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:28.453647 31050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:28.453693 31050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:28.457799 31050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:28.457871 31050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:28.457942 31050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:42:28.463299 31050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:37403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=44413
--tserver_master_addrs=127.8.241.254:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:28.464414 31050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:28.465543 31050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:28.476691 31086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:28.476691 31089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:28.476722 31087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:28.477245 31050 server_base.cc:1034] running on GCE node
I20260806 18:42:28.477787 31050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:28.478615 31050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:28.479830 31050 hybrid_clock.cc:648] HybridClock initialized: now 1786041748479793 us; error 42 us; skew 500 ppm
Aug 06 18:42:28 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:28.483287 31050 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:42:28.484468 31050 webserver.cc:460] Webserver started at http://127.8.241.193:44413/ using document root <none> and password file <none>
I20260806 18:42:28.485261 31050 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:28.485380 31050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:28.490707 31050 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:42:28.493335 31099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:28.494725 31050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:42:28.494832 31050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "b27241ab24d54472b43c1075f1f6ed2f"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "8206aa95fdd04499f5b470c6585e1841"
server_key_iv: "fdbf2ebb48a61c29edcb6b76d11e2220"
server_key_version: "encryptionkey@0"
I20260806 18:42:28.495412 31050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:28.507187 31050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:28.510455 31050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:28.510646 31050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:28.511276 31050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:28.512401 31111 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:42:28.517347 31050 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:42:28.517409 31050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 18:42:28.517468 31050 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:42:28.519199 31050 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:42:28.519245 31050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:42:28.519539 31111 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Bootstrap starting.
I20260806 18:42:28.528211 31050 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:37403
I20260806 18:42:28.528370 31273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:37403 every 8 connection(s)
I20260806 18:42:28.529251 31050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:42:28.537695 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 31050
I20260806 18:42:28.538102 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:36449
--local_ip_for_outbound_sockets=127.8.241.194
--tserver_master_addrs=127.8.241.254:42029
--webserver_port=36773
--webserver_interface=127.8.241.194
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:42:28.540807 31111 log.cc:824] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:42:28 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:28.549125 31111 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:42:28.549237 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.531392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:45965)
0806 18:42:28.532172 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:28.532237 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:28.533413 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:28.534562 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:28.534571 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:28.535087 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:28.535840 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.535862 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.537209 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.537213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.538597 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.538610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.539585 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.539589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.539772 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.541160 (+ 1388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:28.541182 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:28.543022 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:28.545624 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:28.545630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:28.545642 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:28.545999 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:28.546410 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:28.546413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:28.546415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:28.546607 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:28.547375 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:28.547382 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:28.547687 (+ 305us) client_negotiation.cc:769] Sending connection context
0806 18:42:28.547882 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 18:42:28.548220 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":121,"threads_started":1}
I20260806 18:42:28.549566 31111 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Bootstrap complete.
I20260806 18:42:28.549744 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.531437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:45965 (local address 127.8.241.254:42029)
0806 18:42:28.531857 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.531903 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.532389 (+ 486us) server_negotiation.cc:407] Connection header received
0806 18:42:28.533839 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.533851 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.534152 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.534415 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:28.536025 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.536999 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.538751 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.539427 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.539606 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.543276 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:28.543283 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:28.543304 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:28.543343 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:28.545496 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.546127 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.546134 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.546140 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.546307 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.546714 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.546717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.546719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.547246 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:28.547394 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.548046 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.548386 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":122,"threads_started":1}
I20260806 18:42:28.550917 31111 ts_tablet_manager.cc:1397] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.010s
I20260806 18:42:28.552424 31274 heartbeater.cc:347] Connected to a master server at 127.8.241.254:42029
I20260806 18:42:28.552676 31274 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:28.553267 31274 heartbeater.cc:511] Master 127.8.241.254:42029 requested a full tablet report, sending...
I20260806 18:42:28.553686 31111 raft_consensus.cc:348] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:28.554216 31111 raft_consensus.cc:729] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27241ab24d54472b43c1075f1f6ed2f, State: Initialized, Role: FOLLOWER
I20260806 18:42:28.554687 31111 consensus_queue.cc:260] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:28.555583 31111 ts_tablet_manager.cc:1428] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260806 18:42:28.556128 30997 ts_manager.cc:194] Registered new tserver with Master: b27241ab24d54472b43c1075f1f6ed2f (127.8.241.193:37403)
I20260806 18:42:28.560180 30997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:45965
I20260806 18:42:28.563078 31274 heartbeater.cc:503] Master 127.8.241.254:42029 was elected leader, sending a full tablet report...
Aug 06 18:42:28 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:42:28.569916 31289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.559588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.8.241.193:37403)
0806 18:42:28.560026 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.560037 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.560571 (+ 534us) server_negotiation.cc:407] Connection header received
0806 18:42:28.560633 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.560637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.560807 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.560929 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:28.561680 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.562416 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.563287 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.563654 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.563750 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.566158 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:28.566167 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:28.566178 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:28.566202 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:28.567927 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.568465 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.568470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.568471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.568585 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.568845 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.568847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.568848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.569141 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:28.569299 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.569597 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.569752 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":331,"thread_start_us":128,"threads_started":1}
I20260806 18:42:28.569921 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.559485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:37403 (local address 127.0.0.1:36060)
0806 18:42:28.560266 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:28.560315 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:28.560459 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:28.560971 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:28.560977 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:28.561439 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:28.561552 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.561559 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.562537 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.562540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.563159 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.563171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.563773 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.563777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.563883 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.564384 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:28.564396 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:28.566023 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:28.568047 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:28.568057 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:28.568063 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:28.568362 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:28.568665 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:28.568668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:28.568670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:28.568766 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:28.569247 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:28.569253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:28.569385 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 18:42:28.569531 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 18:42:28.569753 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":120,"threads_started":1}
I20260806 18:42:28.571121 31176 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d17f658927d4855815d2c250c52cc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:28 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:36060
I20260806 18:42:28.571986 31290 tablet_replica.cc:330] stopping tablet replica
I20260806 18:42:28.572278 31290 raft_consensus.cc:2229] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:42:28.572535 31290 raft_consensus.cc:2258] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:42:28.573613 31290 ts_tablet_manager.cc:1905] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:42:28.577327 31290 ts_tablet_manager.cc:1918] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:42:28.577415 31290 log.cc:1196] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/0d17f658927d4855815d2c250c52cc82
I20260806 18:42:28.577653 31290 ts_tablet_manager.cc:1939] T 0d17f658927d4855815d2c250c52cc82 P b27241ab24d54472b43c1075f1f6ed2f: Deleting consensus metadata
I20260806 18:42:28.579397 30979 catalog_manager.cc:4841] TS b27241ab24d54472b43c1075f1f6ed2f (127.8.241.193:37403): tablet 0d17f658927d4855815d2c250c52cc82 (table test-table [id=4dfc21a0e83d4ae79020089eea236cc0]) successfully deleted
W20260806 18:42:28.662084 31282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:28.662456 31282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:28.662528 31282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:28.666666 31282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:28.666731 31282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:28.666815 31282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:42:28.672550 31282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:36449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=36773
--tserver_master_addrs=127.8.241.254:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:28.673679 31282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:28.674726 31282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:28.684652 31297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:28.684661 31296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:28.684676 31302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:28.685210 31282 server_base.cc:1034] running on GCE node
I20260806 18:42:28.685638 31282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:28.686287 31282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:28.687464 31282 hybrid_clock.cc:648] HybridClock initialized: now 1786041748687426 us; error 58 us; skew 500 ppm
Aug 06 18:42:28 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:28.690646 31282 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:42:28.691792 31282 webserver.cc:460] Webserver started at http://127.8.241.194:36773/ using document root <none> and password file <none>
I20260806 18:42:28.692525 31282 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:28.692621 31282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:28.699939 31282 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 18:42:28.702425 31315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:28.703768 31282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:42:28.703881 31282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "3041d19d19af49fa84c77d0fa8430903"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "a91359c1d43fcf6f8b41b3d6d9cae651"
server_key_iv: "e9473b497ce51e35985057ea0c0aced7"
server_key_version: "encryptionkey@0"
I20260806 18:42:28.704484 31282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:28.729894 31282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:28.733072 31282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:28.733250 31282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:28.733875 31282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:28.735014 31325 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:42:28.740908 31282 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:42:28.740989 31282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 18:42:28.741081 31282 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:42:28.742877 31282 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:42:28.742926 31282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:42:28.743299 31325 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Bootstrap starting.
I20260806 18:42:28.749872 31325 log.cc:824] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:28.753064 31282 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:36449
I20260806 18:42:28.753171 31477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:36449 every 8 connection(s)
I20260806 18:42:28.753973 31282 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:42:28.756407 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 31282
I20260806 18:42:28.756829 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:34217
--local_ip_for_outbound_sockets=127.8.241.195
--tserver_master_addrs=127.8.241.254:42029
--webserver_port=37945
--webserver_interface=127.8.241.195
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:42:28.759990 31325 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:42:28.760456 31325 tablet_bootstrap.cc:492] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Bootstrap complete.
I20260806 18:42:28.760852 31325 ts_tablet_manager.cc:1397] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Time spent bootstrapping tablet: real 0.018s user 0.017s sys 0.000s
Aug 06 18:42:28 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:28.764513 31325 raft_consensus.cc:348] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:28.765858 31325 raft_consensus.cc:729] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3041d19d19af49fa84c77d0fa8430903, State: Initialized, Role: FOLLOWER
I20260806 18:42:28.766474 31325 consensus_queue.cc:260] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } }
I20260806 18:42:28.768174 31325 ts_tablet_manager.cc:1428] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:42:28.770799 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.755731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:47405 (local address 127.8.241.254:42029)
0806 18:42:28.755883 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.755887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.758403 (+ 2516us) server_negotiation.cc:407] Connection header received
0806 18:42:28.758483 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.758487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.758551 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.758641 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:28.759786 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.760379 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.761840 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.762191 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.762271 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.766268 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:28.766273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:28.766295 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:28.766328 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:28.768040 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.768794 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.768798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.768800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.768904 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.769386 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.769389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.769391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.769645 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:28.769761 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.770464 (+ 703us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.770624 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:42:28.772113 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.755882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:47405)
0806 18:42:28.756394 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:28.756450 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:28.757818 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:28.758812 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:28.758820 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:28.759162 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:28.759629 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.759642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.760529 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.760533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.761692 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.761705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.762455 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.762459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.762670 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.763903 (+ 1233us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:28.763925 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:28.766042 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:28.768197 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:28.768203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:28.768220 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:28.768660 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:28.769027 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:28.769032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:28.769034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:28.769284 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:28.769761 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:28.769768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:28.770115 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 18:42:28.770395 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 18:42:28.770801 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":125,"threads_started":1}
I20260806 18:42:28.773491 31480 heartbeater.cc:347] Connected to a master server at 127.8.241.254:42029
I20260806 18:42:28.773788 31480 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:28.774459 31480 heartbeater.cc:511] Master 127.8.241.254:42029 requested a full tablet report, sending...
I20260806 18:42:28.776623 30997 ts_manager.cc:194] Registered new tserver with Master: 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449)
I20260806 18:42:28.777998 30997 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:47405
I20260806 18:42:28.780560 31480 heartbeater.cc:503] Master 127.8.241.254:42029 was elected leader, sending a full tablet report...
I20260806 18:42:28.792127 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.777670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36449 (local address 127.0.0.1:35138)
0806 18:42:28.777986 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:28.778001 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:28.778087 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:28.788773 (+ 10686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:28.788777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:28.788803 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:28.788915 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.788922 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.790002 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.790006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.790804 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.790815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.791634 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.791638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.791740 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.791798 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 18:42:28.791875 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:42:28.791979 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230}
I20260806 18:42:28.792138 31486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.778267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.8.241.194:36449)
0806 18:42:28.788257 (+ 9990us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.788268 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.788293 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:42:28.788371 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.788376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.788584 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.788750 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:28.789090 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.789871 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.790933 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.791496 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.791601 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.791750 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.791910 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.792000 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9875,"thread_start_us":156,"threads_started":1}
I20260806 18:42:28.793234 31385 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d17f658927d4855815d2c250c52cc82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:28 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:35138
I20260806 18:42:28.794190 31487 tablet_replica.cc:330] stopping tablet replica
I20260806 18:42:28.794498 31487 raft_consensus.cc:2229] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:42:28.794776 31487 raft_consensus.cc:2258] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:42:28.795727 31487 ts_tablet_manager.cc:1905] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:42:28.799652 31487 ts_tablet_manager.cc:1918] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:42:28.799741 31487 log.cc:1196] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/0d17f658927d4855815d2c250c52cc82
I20260806 18:42:28.799978 31487 ts_tablet_manager.cc:1939] T 0d17f658927d4855815d2c250c52cc82 P 3041d19d19af49fa84c77d0fa8430903: Deleting consensus metadata
I20260806 18:42:28.801568 30978 catalog_manager.cc:4841] TS 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449): tablet 0d17f658927d4855815d2c250c52cc82 (table test-table [id=4dfc21a0e83d4ae79020089eea236cc0]) successfully deleted
W20260806 18:42:28.883411 31484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:28.883687 31484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:28.883752 31484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:28.887992 31484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:28.888062 31484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:28.888142 31484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:42:28.893590 31484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:34217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=37945
--tserver_master_addrs=127.8.241.254:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:28.894718 31484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:28.895815 31484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:28.905479 31499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:28.905540 31484 server_base.cc:1034] running on GCE node
W20260806 18:42:28.905560 31494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:28.905707 31495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:28.906149 31484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:28.906798 31484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:28.907951 31484 hybrid_clock.cc:648] HybridClock initialized: now 1786041748907924 us; error 38 us; skew 500 ppm
Aug 06 18:42:28 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:28.910633 31484 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:42:28.911513 31484 webserver.cc:460] Webserver started at http://127.8.241.195:37945/ using document root <none> and password file <none>
I20260806 18:42:28.912084 31484 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:28.912165 31484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:28.916126 31484 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:42:28.918291 31508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:28.919552 31484 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:42:28.919665 31484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "3b5c331882ed41b9a8e582e85d938107"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "6625f7e765544b8530efd82b8b7c1712"
server_key_iv: "ce74319db2eeb83136c207d53c22791e"
server_key_version: "encryptionkey@0"
I20260806 18:42:28.920230 31484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:28.945581 31484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:28.948575 31484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:28.948768 31484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:28.949425 31484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:28.950305 31484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:28.950354 31484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:28.950409 31484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:28.950444 31484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:28.959456 31484 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:34217
I20260806 18:42:28.959474 31668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:34217 every 8 connection(s)
I20260806 18:42:28.960477 31484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:42:28.966375 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 31484
Aug 06 18:42:28 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041748, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:28.974196 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.962317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:42993 (local address 127.8.241.254:42029)
0806 18:42:28.962449 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.962454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.963504 (+ 1050us) server_negotiation.cc:407] Connection header received
0806 18:42:28.964397 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.964401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.964460 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.964535 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:28.965784 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.966407 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.967774 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.968007 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.968097 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.970792 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:28.970795 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:28.970797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:28.970812 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:28.972130 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.972613 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.972617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.972618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.972710 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.973052 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.973055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.973057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.973237 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:28.973328 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.973914 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.974044 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:42:28.975134 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.967043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.8.241.254:42029)
0806 18:42:28.967255 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.967260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.967427 (+ 167us) server_negotiation.cc:407] Connection header received
0806 18:42:28.967555 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.967559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.967602 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.967673 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:28.968406 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.969094 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.970228 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.970489 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.970572 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.972051 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:28.972055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:28.972057 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:28.972101 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:28.973490 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.973914 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.973916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.973918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.973992 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:28.974375 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:28.974378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:28.974380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:28.974557 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:28.974714 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.974920 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.975014 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":60,"threads_started":1}
I20260806 18:42:28.975549 31678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.962694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:42993)
0806 18:42:28.963352 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:28.963394 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:28.964194 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:28.964729 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:28.964736 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:28.965135 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:28.965638 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.965652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.966547 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.966550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.967647 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:28.967659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.968153 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.968179 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:28.968344 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.968950 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:28.968966 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:28.970644 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:28.972239 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:28.972245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:28.972258 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:28.972523 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:28.972793 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:28.972795 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:28.972797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:28.972923 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:28.973343 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:28.973350 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:28.973632 (+ 282us) client_negotiation.cc:769] Sending connection context
0806 18:42:28.973861 (+ 229us) client_negotiation.cc:240] Negotiation successful
0806 18:42:28.974217 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":107,"threads_started":1}
I20260806 18:42:28.976671 31674 heartbeater.cc:347] Connected to a master server at 127.8.241.254:42029
I20260806 18:42:28.976923 31674 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:28.977458 31674 heartbeater.cc:511] Master 127.8.241.254:42029 requested a full tablet report, sending...
I20260806 18:42:28.978482 30996 ts_manager.cc:194] Registered new tserver with Master: 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217)
I20260806 18:42:28.979045 30996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:42993
I20260806 18:42:28.988570 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:42:28.992940 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:28.989178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36886 (local address 127.8.241.254:42029)
0806 18:42:28.989298 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:28.989303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:28.989507 (+ 204us) server_negotiation.cc:407] Connection header received
0806 18:42:28.989679 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:28.989682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:28.989784 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:28.989871 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:42:28.990120 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.990636 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.991526 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:28.991757 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:28.991843 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:28.992154 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:42:28.992558 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:42:28.992664 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:28.992754 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 18:42:28.992814 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:42:28.993553 30996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:42:28.996825 30996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:42:29.004529 31176 tablet_service.cc:1467] Processing CreateTablet for tablet b757b29626874fb0939ac70c93cf0157 (DEFAULT_TABLE table=test-table [id=801d3af556894b9f93a5786c595ac6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:29.004758 31385 tablet_service.cc:1467] Processing CreateTablet for tablet b757b29626874fb0939ac70c93cf0157 (DEFAULT_TABLE table=test-table [id=801d3af556894b9f93a5786c595ac6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:29.005244 31176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b757b29626874fb0939ac70c93cf0157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:29.005333 31385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b757b29626874fb0939ac70c93cf0157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:29.007861 31325 tablet_bootstrap.cc:492] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: Bootstrap starting.
I20260806 18:42:29.007897 31111 tablet_bootstrap.cc:492] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Bootstrap starting.
I20260806 18:42:29.008786 31111 tablet_bootstrap.cc:654] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:29.008895 31325 tablet_bootstrap.cc:654] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:29.010243 31111 tablet_bootstrap.cc:492] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: No bootstrap required, opened a new log
I20260806 18:42:29.010267 31325 tablet_bootstrap.cc:492] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: No bootstrap required, opened a new log
I20260806 18:42:29.010339 31325 ts_tablet_manager.cc:1397] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:42:29.010332 31111 ts_tablet_manager.cc:1397] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:42:29.010780 31111 raft_consensus.cc:348] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.010890 31111 raft_consensus.cc:374] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:29.010923 31111 raft_consensus.cc:729] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27241ab24d54472b43c1075f1f6ed2f, State: Initialized, Role: FOLLOWER
I20260806 18:42:29.010962 31325 raft_consensus.cc:348] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.011063 31325 raft_consensus.cc:374] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:29.011050 31111 consensus_queue.cc:260] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.011101 31325 raft_consensus.cc:729] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3041d19d19af49fa84c77d0fa8430903, State: Initialized, Role: FOLLOWER
I20260806 18:42:29.011291 31111 ts_tablet_manager.cc:1428] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:42:29.011781 31325 consensus_queue.cc:260] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.012513 31325 ts_tablet_manager.cc:1428] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:42:29.012813 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.003782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:34217 (local address 127.0.0.1:57828)
0806 18:42:29.003893 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.003907 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:29.003981 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:29.008712 (+ 4731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:29.008716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:29.008739 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:29.008851 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:29.008859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.010252 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.010255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:29.011314 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:29.011327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.012445 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.012449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:29.012545 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:29.012576 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:42:29.012631 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:42:29.012712 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 18:42:29.013473 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.004138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57828 (local address 127.8.241.195:34217)
0806 18:42:29.008177 (+ 4039us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:29.008191 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:29.008220 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:42:29.008311 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:29.008318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:29.008565 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:29.008741 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:29.009030 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.010115 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.011471 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.012309 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.012994 (+ 685us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:29.013180 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:29.013235 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:42:29.013330 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3914,"thread_start_us":168,"threads_started":1}
I20260806 18:42:29.016444 31577 tablet_service.cc:1467] Processing CreateTablet for tablet b757b29626874fb0939ac70c93cf0157 (DEFAULT_TABLE table=test-table [id=801d3af556894b9f93a5786c595ac6c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:29.017472 31577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b757b29626874fb0939ac70c93cf0157. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:29.024184 31690 tablet_bootstrap.cc:492] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: Bootstrap starting.
I20260806 18:42:29.026628 31690 tablet_bootstrap.cc:654] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:29.027381 31690 log.cc:824] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:29.029364 31690 tablet_bootstrap.cc:492] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: No bootstrap required, opened a new log
I20260806 18:42:29.029569 31690 ts_tablet_manager.cc:1397] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
W20260806 18:42:29.030406 31276 tablet.cc:2369] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:42:29.032527 31690 raft_consensus.cc:348] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.032764 31690 raft_consensus.cc:374] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:29.032815 31690 raft_consensus.cc:729] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5c331882ed41b9a8e582e85d938107, State: Initialized, Role: FOLLOWER
I20260806 18:42:29.033257 31690 consensus_queue.cc:260] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.033902 31674 heartbeater.cc:503] Master 127.8.241.254:42029 was elected leader, sending a full tablet report...
I20260806 18:42:29.034147 31690 ts_tablet_manager.cc:1428] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260806 18:42:29.039599 31485 raft_consensus.cc:482] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:29.039829 31485 raft_consensus.cc:504] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.041206 31485 leader_election.cc:288] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217), b27241ab24d54472b43c1075f1f6ed2f (127.8.241.193:37403)
I20260806 18:42:29.045559 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.041422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:34217 (local address 127.8.241.194:51941)
0806 18:42:29.041647 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.041664 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:29.041764 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:29.042324 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:29.042328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:29.042351 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:29.042504 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:29.042511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.043401 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.043405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:29.044466 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:29.044479 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.045126 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.045130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:29.045242 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:29.045263 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:42:29.045328 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:42:29.045409 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":44}
I20260806 18:42:29.045559 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.041528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:51941 (local address 127.8.241.195:34217)
0806 18:42:29.041674 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:29.041678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:29.041731 (+ 53us) server_negotiation.cc:407] Connection header received
0806 18:42:29.041856 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:29.041859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:29.042193 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:29.042275 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:29.042631 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.043295 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.044606 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.044999 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.045134 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:29.045191 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:29.045355 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 18:42:29.045441 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:42:29.045965 31289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.042004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:39651 (local address 127.8.241.193:37403)
0806 18:42:29.042124 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:29.042128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:29.042312 (+ 184us) server_negotiation.cc:407] Connection header received
0806 18:42:29.042501 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:29.042505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:29.042605 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:29.042753 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:29.043000 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.043575 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.044878 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.045392 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.045509 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:29.045678 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:29.045761 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 18:42:29.045838 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:42:29.045991 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.041554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:37403 (local address 127.8.241.194:39651)
0806 18:42:29.042219 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.042232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:29.042371 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:29.042755 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:29.042759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:29.042779 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:29.042893 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:29.042900 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.043716 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.043720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:29.044752 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:29.044765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.045531 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.045535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:29.045643 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:29.045672 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:42:29.045749 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:42:29.045824 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":1,"thread_start_us":137,"threads_started":1}
I20260806 18:42:29.046373 31604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b757b29626874fb0939ac70c93cf0157" candidate_uuid: "3041d19d19af49fa84c77d0fa8430903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b5c331882ed41b9a8e582e85d938107" is_pre_election: true
I20260806 18:42:29.046500 31208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b757b29626874fb0939ac70c93cf0157" candidate_uuid: "3041d19d19af49fa84c77d0fa8430903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27241ab24d54472b43c1075f1f6ed2f" is_pre_election: true
I20260806 18:42:29.046705 31604 raft_consensus.cc:2454] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3041d19d19af49fa84c77d0fa8430903 in term 0.
I20260806 18:42:29.046752 31208 raft_consensus.cc:2454] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3041d19d19af49fa84c77d0fa8430903 in term 0.
I20260806 18:42:29.047243 31319 leader_election.cc:302] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3041d19d19af49fa84c77d0fa8430903, 3b5c331882ed41b9a8e582e85d938107, b27241ab24d54472b43c1075f1f6ed2f; no voters:
I20260806 18:42:29.047500 31485 raft_consensus.cc:2781] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:42:29.047585 31485 raft_consensus.cc:482] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:42:29.047645 31485 raft_consensus.cc:3037] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:29.048763 31485 raft_consensus.cc:504] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.049152 31485 leader_election.cc:288] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [CANDIDATE]: Term 1 election: Requested vote from peers 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217), b27241ab24d54472b43c1075f1f6ed2f (127.8.241.193:37403)
I20260806 18:42:29.049486 31604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b757b29626874fb0939ac70c93cf0157" candidate_uuid: "3041d19d19af49fa84c77d0fa8430903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b5c331882ed41b9a8e582e85d938107"
I20260806 18:42:29.049486 31208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b757b29626874fb0939ac70c93cf0157" candidate_uuid: "3041d19d19af49fa84c77d0fa8430903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b27241ab24d54472b43c1075f1f6ed2f"
I20260806 18:42:29.049616 31208 raft_consensus.cc:3037] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:29.049616 31604 raft_consensus.cc:3037] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:29.050853 31604 raft_consensus.cc:2454] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3041d19d19af49fa84c77d0fa8430903 in term 1.
I20260806 18:42:29.050853 31208 raft_consensus.cc:2454] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3041d19d19af49fa84c77d0fa8430903 in term 1.
I20260806 18:42:29.051157 31318 leader_election.cc:302] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3041d19d19af49fa84c77d0fa8430903, b27241ab24d54472b43c1075f1f6ed2f; no voters:
I20260806 18:42:29.051327 31485 raft_consensus.cc:2781] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:42:29.051550 31485 raft_consensus.cc:686] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 1 LEADER]: Becoming Leader. State: Replica: 3041d19d19af49fa84c77d0fa8430903, State: Running, Role: LEADER
I20260806 18:42:29.051801 31485 consensus_queue.cc:237] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.054966 30997 catalog_manager.cc:5495] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194). New cstate: current_term: 1 leader_uuid: "3041d19d19af49fa84c77d0fa8430903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } health_report { overall_health: HEALTHY } } }
I20260806 18:42:29.075867 31486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.072792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.8.241.194:36449)
0806 18:42:29.072929 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:29.072934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:29.072950 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:42:29.073083 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:29.073086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:29.073146 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:29.073232 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:42:29.073430 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.073932 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.074675 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:29.074876 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:29.074944 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:29.075125 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:42:29.075589 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:42:29.075674 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:29.075703 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 18:42:29.075753 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:42:29.079990 31208 raft_consensus.cc:1261] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Refusing update from remote peer 3041d19d19af49fa84c77d0fa8430903: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:29.079994 31604 raft_consensus.cc:1261] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [term 1 FOLLOWER]: Refusing update from remote peer 3041d19d19af49fa84c77d0fa8430903: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:29.080703 31699 consensus_queue.cc:1035] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:29.080881 31485 consensus_queue.cc:1035] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:29.087509 31706 mvcc.cc:205] Tried to move back new op lower bound from 7315627004224806912 to 7315627004118769664. Current Snapshot: MvccSnapshot[applied={T|T < 7315627004224806912}]
I20260806 18:42:29.088140 31708 mvcc.cc:205] Tried to move back new op lower bound from 7315627004224806912 to 7315627004118769664. Current Snapshot: MvccSnapshot[applied={T|T < 7315627004224806912}]
I20260806 18:42:29.095496 30996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:42:29.095664 30996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:42:29.097854 30996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Sending DeleteTablet for 3 replicas of tablet b757b29626874fb0939ac70c93cf0157
I20260806 18:42:29.098503 31176 tablet_service.cc:1514] Processing DeleteTablet for tablet b757b29626874fb0939ac70c93cf0157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:29 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:36060
I20260806 18:42:29.098589 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 31050
I20260806 18:42:29.098611 31577 tablet_service.cc:1514] Processing DeleteTablet for tablet b757b29626874fb0939ac70c93cf0157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:29 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:57828
I20260806 18:42:29.105968 31716 tablet_replica.cc:330] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: stopping tablet replica
I20260806 18:42:29.106346 31716 raft_consensus.cc:2229] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:42:29.106699 31716 raft_consensus.cc:2258] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:42:29.107741 31385 tablet_service.cc:1514] Processing DeleteTablet for tablet b757b29626874fb0939ac70c93cf0157 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:29 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:35138
I20260806 18:42:29.107952 31487 tablet_replica.cc:330] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: stopping tablet replica
I20260806 18:42:29.108055 31487 raft_consensus.cc:2229] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:42:29.108373 31487 raft_consensus.cc:2258] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:42:29.108495 31716 ts_tablet_manager.cc:1905] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:42:29.110857 31716 ts_tablet_manager.cc:1918] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:42:29.110988 31716 log.cc:1196] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/b757b29626874fb0939ac70c93cf0157
I20260806 18:42:29.111282 31716 ts_tablet_manager.cc:1939] T b757b29626874fb0939ac70c93cf0157 P 3b5c331882ed41b9a8e582e85d938107: Deleting consensus metadata
I20260806 18:42:29.112557 30980 catalog_manager.cc:4841] TS 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217): tablet b757b29626874fb0939ac70c93cf0157 (table test-table [id=801d3af556894b9f93a5786c595ac6c8]) successfully deleted
I20260806 18:42:29.112851 31487 ts_tablet_manager.cc:1905] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:42:29.114706 31487 ts_tablet_manager.cc:1918] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:42:29.114777 31487 log.cc:1196] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/b757b29626874fb0939ac70c93cf0157
I20260806 18:42:29.114980 31487 ts_tablet_manager.cc:1939] T b757b29626874fb0939ac70c93cf0157 P 3041d19d19af49fa84c77d0fa8430903: Deleting consensus metadata
I20260806 18:42:29.115609 30978 catalog_manager.cc:4841] TS 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449): tablet b757b29626874fb0939ac70c93cf0157 (table test-table [id=801d3af556894b9f93a5786c595ac6c8]) successfully deleted
W20260806 18:42:29.116820 30979 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:37403 (error 108)
I20260806 18:42:29.117450 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 31282
I20260806 18:42:29.117915 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.117611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:37403 (local address 127.0.0.1:36062)
0806 18:42:29.117773 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.117837 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:37403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 18:42:29.118052 30979 catalog_manager.cc:4584] TS b27241ab24d54472b43c1075f1f6ed2f (127.8.241.193:37403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b757b29626874fb0939ac70c93cf0157: Network error: Client connection negotiation failed: client connection to 127.8.241.193:37403: connect: Connection refused (error 111)
I20260806 18:42:29.133491 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 31484
I20260806 18:42:29.154958 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 30948
I20260806 18:42:29.168995 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42029
--webserver_interface=127.8.241.254
--webserver_port=33241
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:29.284685 31718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:29.284960 31718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:29.284998 31718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:29.288959 31718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:29.289042 31718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:29.289062 31718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:29.289078 31718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:29.289103 31718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:29.294452 31718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=33241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:29.295552 31718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:29.296572 31718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:29.305051 31726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:29.305052 31724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:29.305109 31729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:29.305410 31718 server_base.cc:1034] running on GCE node
I20260806 18:42:29.305954 31718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:29.307070 31718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:29.308231 31718 hybrid_clock.cc:648] HybridClock initialized: now 1786041749308211 us; error 36 us; skew 500 ppm
Aug 06 18:42:29 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:29.311236 31718 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:42:29.312151 31718 webserver.cc:460] Webserver started at http://127.8.241.254:33241/ using document root <none> and password file <none>
I20260806 18:42:29.312695 31718 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:29.312753 31718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:29.316741 31718 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:42:29.318914 31738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:29.320144 31718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:42:29.320252 31718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "0ca7f856b4d840799a89192870b79d0a"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "98ce906678c789243f2e4d984e5576ee"
server_key_iv: "d09aa25483481852d877ae4a0b471a66"
server_key_version: "encryptionkey@0"
I20260806 18:42:29.320793 31718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:29.331032 31718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:29.333775 31718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:29.333930 31718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:29.340730 31718 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:42029
I20260806 18:42:29.340993 31817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:42029 every 8 connection(s)
I20260806 18:42:29.341863 31718 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:42:29.345319 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 31718
I20260806 18:42:29.345741 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:37403
--local_ip_for_outbound_sockets=127.8.241.193
--tserver_master_addrs=127.8.241.254:42029
--webserver_port=44413
--webserver_interface=127.8.241.193
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:42:29.346374 31819 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:42:29.348416 31819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Bootstrap starting.
I20260806 18:42:29.366900 31819 log.cc:824] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:29.378604 31819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:42:29.379012 31819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a: Bootstrap complete.
I20260806 18:42:29.383340 31819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:29.383769 31819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ca7f856b4d840799a89192870b79d0a, State: Initialized, Role: FOLLOWER
I20260806 18:42:29.384287 31819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:29.384413 31819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:42:29.384487 31819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:42:29.384580 31819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:42:29.386494 31819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:29.386812 31819 leader_election.cc:302] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ca7f856b4d840799a89192870b79d0a; no voters:
I20260806 18:42:29.387113 31819 leader_election.cc:288] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:42:29.387221 31823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:42:29.387419 31823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [term 3 LEADER]: Becoming Leader. State: Replica: 0ca7f856b4d840799a89192870b79d0a, State: Running, Role: LEADER
I20260806 18:42:29.387713 31823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } }
I20260806 18:42:29.388391 31819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:29.389153 31824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0ca7f856b4d840799a89192870b79d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } } }
I20260806 18:42:29.389281 31824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:29.389591 31825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ca7f856b4d840799a89192870b79d0a. Latest consensus state: current_term: 3 leader_uuid: "0ca7f856b4d840799a89192870b79d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ca7f856b4d840799a89192870b79d0a" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42029 } } }
I20260806 18:42:29.389750 31825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ca7f856b4d840799a89192870b79d0a [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:29.390062 31839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:42:29.395641 31839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:42:29.396588 31839 catalog_manager.cc:1259] Loaded cluster ID: 7f54009a3bdc40e5a4d1f279804ad244
I20260806 18:42:29.396656 31839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 18:42:29.397828 31839 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc7b1a599ad google::LogMessage::Fail() at ??:0
@ 0x7fc7b1a5c88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc7b1a594de google::LogMessage::Flush() at ??:0
@ 0x7fc7b1a5aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc7bc398fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fc7bc399d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc7bc396579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fc7bc3da640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc7b3cab3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc7b3cabc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fc7b3cad3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fc7b3c9c512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc7b52e06db start_thread at ??:0
@ 0x7fc7b055761f clone at ??:0
W20260806 18:42:29.481918 31821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:29.482265 31821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:29.482332 31821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:29.487224 31821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:29.487300 31821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:29.487387 31821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:42:29.492955 31821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:37403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=44413
--tserver_master_addrs=127.8.241.254:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:29.494125 31821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:29.495165 31821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:29.505139 31886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:29.505134 31887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:29.505645 31821 server_base.cc:1034] running on GCE node
W20260806 18:42:29.505741 31891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:29.506306 31821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:29.507164 31821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:29.508378 31821 hybrid_clock.cc:648] HybridClock initialized: now 1786041749508335 us; error 51 us; skew 500 ppm
Aug 06 18:42:29 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:29.511878 31821 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:42:29.513072 31821 webserver.cc:460] Webserver started at http://127.8.241.193:44413/ using document root <none> and password file <none>
I20260806 18:42:29.513813 31821 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:29.513919 31821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:29.519331 31821 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 18:42:29.522192 31901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:29.523720 31821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:42:29.523882 31821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "b27241ab24d54472b43c1075f1f6ed2f"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "8206aa95fdd04499f5b470c6585e1841"
server_key_iv: "fdbf2ebb48a61c29edcb6b76d11e2220"
server_key_version: "encryptionkey@0"
I20260806 18:42:29.524614 31821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:29.550367 31821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:29.554198 31821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:29.554410 31821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:29.555173 31821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:29.556593 31909 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:42:29.561107 31821 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:42:29.561189 31821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260806 18:42:29.561267 31821 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:42:29.563608 31821 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:42:29.563665 31821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:42:29.563995 31909 tablet_bootstrap.cc:492] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Bootstrap starting.
I20260806 18:42:29.574723 31821 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:37403
I20260806 18:42:29.574771 32071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:37403 every 8 connection(s)
I20260806 18:42:29.575937 31821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:42:29.583921 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 31821
I20260806 18:42:29.584409 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:36449
--local_ip_for_outbound_sockets=127.8.241.194
--tserver_master_addrs=127.8.241.254:42029
--webserver_port=36773
--webserver_interface=127.8.241.194
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:42:29.584611 31909 log.cc:824] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:29.592941 31909 tablet_bootstrap.cc:492] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:42:29.593393 31909 tablet_bootstrap.cc:492] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Bootstrap complete.
I20260806 18:42:29.594940 31909 ts_tablet_manager.cc:1397] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Time spent bootstrapping tablet: real 0.031s user 0.015s sys 0.013s
I20260806 18:42:29.598461 31909 raft_consensus.cc:348] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.599058 31909 raft_consensus.cc:729] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b27241ab24d54472b43c1075f1f6ed2f, State: Initialized, Role: FOLLOWER
I20260806 18:42:29.599638 31909 consensus_queue.cc:260] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:29.600733 31909 ts_tablet_manager.cc:1428] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
W20260806 18:42:29.631197 32075 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:42:29.578416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:59775)
0806 18:42:29.579062 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.579126 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:29.580319 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:29.630365 (+ 50046us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":167,"threads_started":1}
W20260806 18:42:29.631711 32073 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:42029 (0 consecutive failures): Network error: Failed to ping master at 127.8.241.254:42029: Client connection negotiation failed: client connection to 127.8.241.254:42029: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:42:29.632460 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.632214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:34733)
0806 18:42:29.632336 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.632404 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 18:42:29.633092 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.632862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:35471)
0806 18:42:29.632961 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.633034 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:42:29.633234 32073 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:42:29.711644 32077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:29.711936 32077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:29.712006 32077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:29.716101 32077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:29.716176 32077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:29.716259 32077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:42:29.721776 32077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:36449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=36773
--tserver_master_addrs=127.8.241.254:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:29.722836 32077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:29.723879 32077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:29.733692 32084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:29.733798 32085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:29.733785 32077 server_base.cc:1034] running on GCE node
W20260806 18:42:29.733700 32089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:29.734475 32077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:29.735090 32077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:29.736248 32077 hybrid_clock.cc:648] HybridClock initialized: now 1786041749736223 us; error 36 us; skew 500 ppm
Aug 06 18:42:29 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:29.738937 32077 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:42:29.739807 32077 webserver.cc:460] Webserver started at http://127.8.241.194:36773/ using document root <none> and password file <none>
I20260806 18:42:29.740340 32077 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:29.740399 32077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:29.744228 32077 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:42:29.746344 32097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:29.747591 32077 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:42:29.747700 32077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "3041d19d19af49fa84c77d0fa8430903"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "a91359c1d43fcf6f8b41b3d6d9cae651"
server_key_iv: "e9473b497ce51e35985057ea0c0aced7"
server_key_version: "encryptionkey@0"
I20260806 18:42:29.748281 32077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:29.757756 32077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:29.760808 32077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:29.760998 32077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:29.761602 32077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:29.762457 32077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:29.762506 32077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:29.762552 32077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:29.762581 32077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:29.771415 32077 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:36449
I20260806 18:42:29.771440 32268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:36449 every 8 connection(s)
I20260806 18:42:29.772459 32077 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:42:29.773952 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 32077
I20260806 18:42:29.774475 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:34217
--local_ip_for_outbound_sockets=127.8.241.195
--tserver_master_addrs=127.8.241.254:42029
--webserver_port=37945
--webserver_interface=127.8.241.195
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:42:29.776626 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.774897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:52665)
0806 18:42:29.775631 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.775747 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":131,"threads_started":1}
W20260806 18:42:29.777103 32269 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:42029 (0 consecutive failures): Network error: Failed to ping master at 127.8.241.254:42029: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
I20260806 18:42:29.777707 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.777471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:52709)
0806 18:42:29.777587 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.777643 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 18:42:29.778275 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.778080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:52007)
0806 18:42:29.778180 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.778226 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:42:29.778422 32269 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:42:29.889479 32273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:29.889739 32273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:29.889784 32273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:29.893801 32273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:29.893864 32273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:29.893934 32273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:42:29.899150 32273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:34217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=37945
--tserver_master_addrs=127.8.241.254:42029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:29.900264 32273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:29.901348 32273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:29.910967 32282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:29.910985 32283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:29.910982 32285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:29.911283 32273 server_base.cc:1034] running on GCE node
I20260806 18:42:29.911789 32273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:29.912391 32273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:29.913566 32273 hybrid_clock.cc:648] HybridClock initialized: now 1786041749913535 us; error 40 us; skew 500 ppm
Aug 06 18:42:29 dist-test-slave-k5rr krb5kdc[30144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:29.916126 32273 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:42:29.917039 32273 webserver.cc:460] Webserver started at http://127.8.241.195:37945/ using document root <none> and password file <none>
I20260806 18:42:29.917656 32273 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:29.917726 32273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:29.921639 32273 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:42:29.923707 32294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:29.924932 32273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:42:29.925079 32273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "3b5c331882ed41b9a8e582e85d938107"
format_stamp: "Formatted at 2026-08-06 18:42:27 on dist-test-slave-k5rr"
server_key: "6625f7e765544b8530efd82b8b7c1712"
server_key_iv: "ce74319db2eeb83136c207d53c22791e"
server_key_version: "encryptionkey@0"
I20260806 18:42:29.925642 32273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:29.948460 32273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:29.951372 32273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:29.951604 32273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:29.952239 32273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:29.953166 32273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:29.953214 32273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:29.953260 32273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:29.953289 32273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:29.962467 32273 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:34217
I20260806 18:42:29.962496 32462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:34217 every 8 connection(s)
I20260806 18:42:29.963538 32273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:42:29.967322 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.965825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:41119)
0806 18:42:29.966505 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.966618 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":122,"threads_started":1}
W20260806 18:42:29.967777 32463 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:42029 (0 consecutive failures): Network error: Failed to ping master at 127.8.241.254:42029: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
I20260806 18:42:29.968381 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.968180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:41463)
0806 18:42:29.968279 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.968326 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:42:29.968997 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:29.968798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:56231)
0806 18:42:29.968897 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:29.968943 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:42:29.969194 32463 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:42:29.971953 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 32273
I20260806 18:42:30.634325 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:30.633839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:34991)
0806 18:42:30.634127 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:30.634195 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":100,"threads_started":1}
I20260806 18:42:30.779394 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:30.778963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:38481)
0806 18:42:30.779222 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:30.779281 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":72,"threads_started":1}
I20260806 18:42:30.888696 32471 raft_consensus.cc:482] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:30.888924 32471 raft_consensus.cc:504] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:30.890115 32471 leader_election.cc:288] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217), 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449)
Aug 06 18:42:30 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.195@KRBTEST.COM
Aug 06 18:42:30 dist-test-slave-k5rr krb5kdc[30144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041749, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.194@KRBTEST.COM
I20260806 18:42:30.903872 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:30.890334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:34217 (local address 127.8.241.193:54561)
0806 18:42:30.890529 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:30.890544 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:30.890651 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:30.892282 (+ 1631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:30.892288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:30.892829 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:30.893347 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:30.893361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:30.894636 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:30.894639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:30.896201 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:30.896213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:30.896902 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:30.896905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:30.897105 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:30.897705 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:30.897776 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:30.899373 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:30.901499 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:30.901506 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:30.901521 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:30.901869 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:30.902221 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:30.902224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:30.902226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:30.902374 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:30.903016 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:30.903019 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:30.903296 (+ 277us) client_negotiation.cc:769] Sending connection context
0806 18:42:30.903456 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 18:42:30.903716 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":209,"spinlock_wait_cycles":24320}
I20260806 18:42:30.904071 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:30.890493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:54561 (local address 127.8.241.195:34217)
0806 18:42:30.890796 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:30.890814 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:30.890854 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:42:30.891554 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:30.891561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:30.891918 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:30.892176 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:30.893528 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:30.894498 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:30.896318 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:30.896794 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:30.896924 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:30.899634 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:30.899641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:30.899659 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:30.899693 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:30.901369 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:30.901995 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:30.902002 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:30.902007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:30.902126 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:30.902467 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:30.902471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:30.902473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:30.902868 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:30.903128 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:30.903609 (+ 481us) server_negotiation.cc:299] Negotiation successful
0806 18:42:30.903892 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
I20260806 18:42:30.904302 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:30.890369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36449 (local address 127.8.241.193:34807)
0806 18:42:30.890880 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:30.890894 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:30.890989 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:30.892283 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:30.892289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:30.892829 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:30.893346 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:30.893361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:30.894676 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:30.894679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:30.896203 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:30.896213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:30.897824 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:30.897828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:30.897917 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:30.898300 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:30.898314 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:30.899702 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:30.901693 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:30.901696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:30.901697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:30.901901 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:30.902226 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:30.902229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:30.902231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:30.902370 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:30.902999 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:30.903006 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:30.903291 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 18:42:30.903467 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 18:42:30.903806 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":334,"spinlock_wait_cycles":60672,"thread_start_us":85,"threads_started":1}
I20260806 18:42:30.905606 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:30.890536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:34807 (local address 127.8.241.194:36449)
0806 18:42:30.890931 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:30.890968 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:30.891009 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:42:30.891684 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:30.891690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:30.891987 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:30.892256 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:30.893512 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:30.894547 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:30.897107 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:30.897691 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:30.897807 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:30.899876 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:30.899880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:30.899891 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:30.899920 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:30.901586 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:30.902008 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:30.902012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:30.902016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:30.902130 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:30.902462 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:30.902465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:30.902467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:30.902868 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:30.902998 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:30.905149 (+ 2151us) server_negotiation.cc:299] Negotiation successful
0806 18:42:30.905433 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":85,"threads_started":1}
W20260806 18:42:30.906699 31904 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
W20260806 18:42:30.907891 31902 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
I20260806 18:42:30.907981 31902 leader_election.cc:302] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b27241ab24d54472b43c1075f1f6ed2f; no voters: 3041d19d19af49fa84c77d0fa8430903, 3b5c331882ed41b9a8e582e85d938107
I20260806 18:42:30.908229 32471 raft_consensus.cc:2726] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:42:30.970151 32476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:30.969728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:44349)
0806 18:42:30.969989 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:30.970050 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":87,"threads_started":1}
I20260806 18:42:31.635388 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:31.634984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:46633)
0806 18:42:31.635234 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:31.635290 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
I20260806 18:42:31.780618 32480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:31.780187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:46015)
0806 18:42:31.780467 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:31.780524 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":85,"threads_started":1}
I20260806 18:42:31.971226 32481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:31.970821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:40845)
0806 18:42:31.971061 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:31.971114 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":72,"threads_started":1}
I20260806 18:42:32.636582 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:32.636122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:54441)
0806 18:42:32.636403 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:32.636476 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":87,"threads_started":1}
I20260806 18:42:32.728221 32483 raft_consensus.cc:482] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:32.728389 32483 raft_consensus.cc:504] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:32.728796 32483 leader_election.cc:288] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217), 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449)
W20260806 18:42:32.729705 31904 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
W20260806 18:42:32.729840 31902 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
I20260806 18:42:32.729939 31902 leader_election.cc:302] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b27241ab24d54472b43c1075f1f6ed2f; no voters: 3041d19d19af49fa84c77d0fa8430903, 3b5c331882ed41b9a8e582e85d938107
I20260806 18:42:32.730094 32483 raft_consensus.cc:2726] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:42:32.781827 32485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:32.781365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:34571)
0806 18:42:32.781647 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:32.781709 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":96,"threads_started":1}
I20260806 18:42:32.972326 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:32.971928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:38111)
0806 18:42:32.972168 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:32.972226 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":79,"threads_started":1}
I20260806 18:42:33.637916 32487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:33.637459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:41287)
0806 18:42:33.637744 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:33.637809 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260806 18:42:33.782941 32488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:33.782551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:34021)
0806 18:42:33.782786 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:33.782844 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":75,"threads_started":1}
I20260806 18:42:33.973533 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:33.973082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:53237)
0806 18:42:33.973359 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:33.973418 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":95,"threads_started":1}
I20260806 18:42:34.344429 32491 raft_consensus.cc:482] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:34.344575 32491 raft_consensus.cc:504] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:34.344995 32491 leader_election.cc:288] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217), 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449)
W20260806 18:42:34.345793 31902 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
W20260806 18:42:34.345908 31904 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
I20260806 18:42:34.346016 31904 leader_election.cc:302] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b27241ab24d54472b43c1075f1f6ed2f; no voters: 3041d19d19af49fa84c77d0fa8430903, 3b5c331882ed41b9a8e582e85d938107
I20260806 18:42:34.346140 32491 raft_consensus.cc:2726] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:42:34.639479 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:34.638987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:57817)
0806 18:42:34.639309 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:34.639374 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":79,"threads_started":1}
I20260806 18:42:34.784227 32493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:34.783762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:53029)
0806 18:42:34.784063 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:34.784128 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":97,"threads_started":1}
I20260806 18:42:34.974711 32496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:34.974275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:40509)
0806 18:42:34.974555 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:34.974617 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":73,"threads_started":1}
I20260806 18:42:35.640637 32498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:35.640204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:57225)
0806 18:42:35.640476 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:35.640536 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":75,"threads_started":1}
I20260806 18:42:35.785430 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:35.784993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:38627)
0806 18:42:35.785266 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:35.785326 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
I20260806 18:42:35.975999 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:35.975536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:34791)
0806 18:42:35.975826 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:35.975885 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":93,"threads_started":1}
I20260806 18:42:36.641970 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:36.641463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:39959)
0806 18:42:36.641769 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:36.641850 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":92,"threads_started":1}
I20260806 18:42:36.786612 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:36.786146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:41421)
0806 18:42:36.786460 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:36.786515 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":95,"threads_started":1}
I20260806 18:42:36.977442 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:36.976880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:33105)
0806 18:42:36.977272 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:36.977334 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":140,"threads_started":1}
I20260806 18:42:37.643173 32513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:37.642712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:40301)
0806 18:42:37.643010 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:37.643070 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":82,"threads_started":1}
I20260806 18:42:37.787832 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:37.787391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:41453)
0806 18:42:37.787665 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:37.787726 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
I20260806 18:42:37.978781 32515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:37.978303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:35669)
0806 18:42:37.978609 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:37.978677 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":90,"threads_started":1}
I20260806 18:42:38.485574 32519 raft_consensus.cc:482] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:38.485728 32519 raft_consensus.cc:504] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:38.486194 32519 leader_election.cc:288] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217), 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449)
W20260806 18:42:38.487025 31902 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
W20260806 18:42:38.487128 31904 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
I20260806 18:42:38.487237 31904 leader_election.cc:302] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b27241ab24d54472b43c1075f1f6ed2f; no voters: 3041d19d19af49fa84c77d0fa8430903, 3b5c331882ed41b9a8e582e85d938107
I20260806 18:42:38.487392 32519 raft_consensus.cc:2726] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:42:38.644345 32520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:38.643882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:41235)
0806 18:42:38.644181 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:38.644239 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
I20260806 18:42:38.789144 32521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:38.788641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:48253)
0806 18:42:38.788910 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:38.788971 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
I20260806 18:42:38.980062 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:38.979618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:43709)
0806 18:42:38.979899 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:38.979958 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":105,"threads_started":1}
I20260806 18:42:39.645664 32523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:39.645228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:59647)
0806 18:42:39.645497 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:39.645560 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260806 18:42:39.790277 32524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:39.789836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:37097)
0806 18:42:39.790121 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:39.790177 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":115,"threads_started":1}
I20260806 18:42:39.981411 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:39.980950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:55189)
0806 18:42:39.981243 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:39.981307 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":113,"threads_started":1}
I20260806 18:42:40.646854 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:40.646390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:39689)
0806 18:42:40.646688 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:40.646750 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
I20260806 18:42:40.791528 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:40.791056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:47895)
0806 18:42:40.791325 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:40.791389 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260806 18:42:40.982570 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:40.982122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:54583)
0806 18:42:40.982392 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:40.982466 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":91,"threads_started":1}
I20260806 18:42:41.648065 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:41.647614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:43251)
0806 18:42:41.647890 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:41.647960 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":84,"threads_started":1}
I20260806 18:42:41.792871 32530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:41.792376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:50149)
0806 18:42:41.792689 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:41.792758 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
I20260806 18:42:41.983743 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:41.983283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:45411)
0806 18:42:41.983573 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:41.983637 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":96,"threads_started":1}
I20260806 18:42:42.649338 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:42.648833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:35897)
0806 18:42:42.649169 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:42.649233 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
I20260806 18:42:42.794100 32535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:42.793612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:39105)
0806 18:42:42.793919 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:42.793995 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":97,"threads_started":1}
I20260806 18:42:42.985201 32536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:42.984653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:55845)
0806 18:42:42.984993 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:42.985094 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":112,"threads_started":1}
I20260806 18:42:43.650439 32537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:43.650041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:56893)
0806 18:42:43.650279 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:43.650334 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":75,"threads_started":1}
I20260806 18:42:43.795301 32539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:43.794865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:34057)
0806 18:42:43.795136 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:43.795193 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":106,"threads_started":1}
I20260806 18:42:43.986361 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:43.985934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:56487)
0806 18:42:43.986191 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:43.986257 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
I20260806 18:42:44.618103 32545 raft_consensus.cc:482] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:44.618247 32545 raft_consensus.cc:504] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b5c331882ed41b9a8e582e85d938107" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 34217 } } peers { permanent_uuid: "b27241ab24d54472b43c1075f1f6ed2f" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 37403 } } peers { permanent_uuid: "3041d19d19af49fa84c77d0fa8430903" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36449 } }
I20260806 18:42:44.618681 32545 leader_election.cc:288] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217), 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449)
W20260806 18:42:44.619529 31904 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3b5c331882ed41b9a8e582e85d938107 (127.8.241.195:34217): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
W20260806 18:42:44.619648 31902 leader_election.cc:341] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3041d19d19af49fa84c77d0fa8430903 (127.8.241.194:36449): Not found: Tablet not found: b757b29626874fb0939ac70c93cf0157
I20260806 18:42:44.619715 31902 leader_election.cc:302] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b27241ab24d54472b43c1075f1f6ed2f; no voters: 3041d19d19af49fa84c77d0fa8430903, 3b5c331882ed41b9a8e582e85d938107
I20260806 18:42:44.619849 32545 raft_consensus.cc:2726] T b757b29626874fb0939ac70c93cf0157 P b27241ab24d54472b43c1075f1f6ed2f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:42:44.651835 32546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:44.651407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.193:44705)
0806 18:42:44.651680 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:44.651736 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":93,"threads_started":1}
I20260806 18:42:44.796561 32547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:44.796121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.194:58795)
0806 18:42:44.796388 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:44.796459 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":93,"threads_started":1}
I20260806 18:42:44.978576 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 31821
I20260806 18:42:44.989216 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:44.987952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:42029 (local address 127.8.241.195:35593)
0806 18:42:44.989026 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:44.989097 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":952,"thread_start_us":361,"threads_started":1}
I20260806 18:42:44.997396 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 32077
I20260806 18:42:45.017199 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 32273
2026-08-06T18:42:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19383 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:42:45 dist-test-slave-k5rr krb5kdc[32555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:42:45 dist-test-slave-k5rr krb5kdc[32555](info): set up 2 sockets
Aug 06 18:42:45 dist-test-slave-k5rr krb5kdc[32555](info): commencing operation
krb5kdc: starting...
W20260806 18:42:46.158289 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:42:46 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:42:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:42:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:42:46.281764 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39103
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:46.398357 32580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:46.398625 32580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:46.398666 32580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:46.402595 32580 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:42:46.402644 32580 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:42:46.402658 32580 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:42:46.402670 32580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:42:46.402704 32580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:46.402734 32580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:46.402761 32580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:42:46.402782 32580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:42:46.408097 32580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:46.409147 32580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:46.410167 32580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:46.418896 32595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:46.418953 32587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:46.419034 32580 server_base.cc:1034] running on GCE node
W20260806 18:42:46.418967 32593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:46.419705 32580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:46.420797 32580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:46.421991 32580 hybrid_clock.cc:648] HybridClock initialized: now 1786041766421949 us; error 49 us; skew 500 ppm
Aug 06 18:42:46 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041766, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:46.424718 32580 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:42:46.425710 32580 webserver.cc:460] Webserver started at http://127.8.241.254:33977/ using document root <none> and password file <none>
I20260806 18:42:46.426280 32580 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:46.426348 32580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:46.426558 32580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:46.428647 32580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "a9aafb8d917a40218b9f38a792af6d31"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "62b20c342a74bde5d35c7b5d9151e64e"
server_key_iv: "45a939d8ef29cb01dac82f8d4cf5fb92"
server_key_version: "encryptionkey@0"
I20260806 18:42:46.429270 32580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "a9aafb8d917a40218b9f38a792af6d31"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "62b20c342a74bde5d35c7b5d9151e64e"
server_key_iv: "45a939d8ef29cb01dac82f8d4cf5fb92"
server_key_version: "encryptionkey@0"
I20260806 18:42:46.432991 32580 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:42:46.435379 32604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:46.436584 32580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:42:46.436700 32580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "a9aafb8d917a40218b9f38a792af6d31"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "62b20c342a74bde5d35c7b5d9151e64e"
server_key_iv: "45a939d8ef29cb01dac82f8d4cf5fb92"
server_key_version: "encryptionkey@0"
I20260806 18:42:46.436798 32580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:46.464175 32580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:46.466943 32580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:46.467121 32580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:46.474323 32580 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:39103
I20260806 18:42:46.474591 32685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:39103 every 8 connection(s)
I20260806 18:42:46.475308 32580 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:42:46.478247 32686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:46.479156 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 32580
I20260806 18:42:46.479274 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:42:46.479604 9159 external_mini_cluster.cc:1428] Setting key 4898261e005e97cff9765177bb7bcc64
I20260806 18:42:46.484894 32686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Bootstrap starting.
I20260806 18:42:46.487792 32686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Neither blocks nor log segments found. Creating new log.
Aug 06 18:42:46 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:46.488662 32686 log.cc:824] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:46.490592 32686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: No bootstrap required, opened a new log
I20260806 18:42:46.493697 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:46.481078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.8.241.254:39103)
0806 18:42:46.481468 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:46.481484 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:46.481519 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:42:46.482186 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:46.482202 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:46.482541 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:46.482820 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:46.483741 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.484723 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.485901 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.486381 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.486515 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:46.489295 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:46.489301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:46.489316 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:46.489346 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:46.490953 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:46.491362 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:46.491368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:46.491373 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:46.491499 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:46.491766 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:46.491769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:46.491771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:46.492168 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:46.492350 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:46.492651 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 18:42:46.492944 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":96,"threads_started":1}
I20260806 18:42:46.494344 32686 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:42:46.494580 32686 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:46.494670 32686 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9aafb8d917a40218b9f38a792af6d31, State: Initialized, Role: FOLLOWER
I20260806 18:42:46.495163 32686 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:42:46.495332 32686 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:42:46.495395 32686 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:42:46.495499 32686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:46.496507 32686 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:42:46.496817 32686 leader_election.cc:302] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9aafb8d917a40218b9f38a792af6d31; no voters:
I20260806 18:42:46.497109 32686 leader_election.cc:288] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:42:46.497203 32693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:42:46.497427 32693 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 1 LEADER]: Becoming Leader. State: Replica: a9aafb8d917a40218b9f38a792af6d31, State: Running, Role: LEADER
I20260806 18:42:46.497767 32693 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:42:46.498323 32686 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:42:46.499562 32695 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9aafb8d917a40218b9f38a792af6d31. Latest consensus state: current_term: 1 leader_uuid: "a9aafb8d917a40218b9f38a792af6d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } } }
I20260806 18:42:46.499687 32695 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:46.499972 32694 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9aafb8d917a40218b9f38a792af6d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } } }
I20260806 18:42:46.500062 32694 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: This master's current role is: LEADER
I20260806 18:42:46.500126 32702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:42:46.502919 32702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:42:46.508628 32702 catalog_manager.cc:1347] Generated new cluster ID: 4a43eb72728d421aaa81913a02fedd50
I20260806 18:42:46.508695 32702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:42:46.528681 32702 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:42:46.530064 32702 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:42:46.535356 32702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Generated new TSK 0
I20260806 18:42:46.536168 32702 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:42:46.569819 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39103
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:46.686247 32723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:46.686525 32723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:46.686573 32723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:46.690584 32723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:46.690645 32723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:46.690716 32723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:42:46.696319 32723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:46.697494 32723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:46.698561 32723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:46.708642 32730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:46.708624 32729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:46.708624 32736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:46.708734 32723 server_base.cc:1034] running on GCE node
I20260806 18:42:46.709331 32723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:46.710001 32723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:46.711177 32723 hybrid_clock.cc:648] HybridClock initialized: now 1786041766711153 us; error 32 us; skew 500 ppm
Aug 06 18:42:46 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041766, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:46.713989 32723 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:42:46.714895 32723 webserver.cc:460] Webserver started at http://127.8.241.193:40949/ using document root <none> and password file <none>
I20260806 18:42:46.715449 32723 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:46.715513 32723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:46.715720 32723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:46.717885 32723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "181bfe4e15e145b39095c438caccc52b"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "3b692be522385de50bdaacca05b52d9b"
server_key_iv: "905e38dfedd112d9318b7fae9c6cfc13"
server_key_version: "encryptionkey@0"
I20260806 18:42:46.718465 32723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "181bfe4e15e145b39095c438caccc52b"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "3b692be522385de50bdaacca05b52d9b"
server_key_iv: "905e38dfedd112d9318b7fae9c6cfc13"
server_key_version: "encryptionkey@0"
I20260806 18:42:46.722405 32723 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:42:46.724754 32743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:46.726037 32723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:42:46.726150 32723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "181bfe4e15e145b39095c438caccc52b"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "3b692be522385de50bdaacca05b52d9b"
server_key_iv: "905e38dfedd112d9318b7fae9c6cfc13"
server_key_version: "encryptionkey@0"
I20260806 18:42:46.726249 32723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:46.742092 32723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:46.744911 32723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:46.745114 32723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:46.745702 32723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:46.746537 32723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:46.746587 32723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:46.746627 32723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:46.746647 32723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:46.755527 32723 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:40419
I20260806 18:42:46.755599 440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:40419 every 8 connection(s)
I20260806 18:42:46.756441 32723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:42:46.756699 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 32723
I20260806 18:42:46.756808 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:42:46.757150 9159 external_mini_cluster.cc:1428] Setting key 114301cf081277cf21f086e02f9f07b1
Aug 06 18:42:46 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041766, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:42:46.771787 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:46.758376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:41463 (local address 127.8.241.254:39103)
0806 18:42:46.758490 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:46.758494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:46.759666 (+ 1172us) server_negotiation.cc:407] Connection header received
0806 18:42:46.761348 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:46.761351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:46.761406 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:46.761489 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:46.762830 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.763355 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.765100 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.765332 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.765413 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:46.768292 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:46.768296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:46.768297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:46.768324 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:46.769644 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:46.770117 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:46.770119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:46.770121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:46.770194 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:46.770530 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:46.770532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:46.770533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:46.770802 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:46.770905 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:46.771533 (+ 628us) server_negotiation.cc:299] Negotiation successful
0806 18:42:46.771656 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:42:46.772924 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:46.758946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:41463)
0806 18:42:46.759483 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:46.759543 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:46.761137 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:46.761658 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:46.761665 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:46.762034 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:46.762672 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:46.762692 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.763507 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.763511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:46.764943 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:46.764956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.765500 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.765504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:46.765717 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:46.766322 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:46.766343 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:46.768077 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:46.769744 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:46.769749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:46.769763 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:46.770023 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:46.770308 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:46.770310 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:46.770312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:46.770459 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:46.770919 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:46.770927 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:46.771228 (+ 301us) client_negotiation.cc:769] Sending connection context
0806 18:42:46.771467 (+ 239us) client_negotiation.cc:240] Negotiation successful
0806 18:42:46.771834 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":125,"threads_started":1}
I20260806 18:42:46.774245 441 heartbeater.cc:347] Connected to a master server at 127.8.241.254:39103
I20260806 18:42:46.774526 441 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:46.775152 441 heartbeater.cc:511] Master 127.8.241.254:39103 requested a full tablet report, sending...
I20260806 18:42:46.776855 32627 ts_manager.cc:194] Registered new tserver with Master: 181bfe4e15e145b39095c438caccc52b (127.8.241.193:40419)
I20260806 18:42:46.778054 32627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:41463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:42:46.789206 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39103
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:46.903451 446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:46.903703 446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:46.903774 446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:46.907648 446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:46.907701 446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:46.907786 446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:42:46.912834 446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:46.913836 446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:46.914824 446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:46.924368 455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:46.924378 456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:46.924414 446 server_base.cc:1034] running on GCE node
W20260806 18:42:46.924383 458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:46.925194 446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:46.925835 446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:46.926999 446 hybrid_clock.cc:648] HybridClock initialized: now 1786041766926977 us; error 36 us; skew 500 ppm
Aug 06 18:42:46 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041766, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:46.929616 446 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:42:46.930495 446 webserver.cc:460] Webserver started at http://127.8.241.194:35117/ using document root <none> and password file <none>
I20260806 18:42:46.931053 446 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:46.931118 446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:46.931331 446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:46.933400 446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "27db750cc47c45beb5690eb887b5b57a"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "73f8d8e7c4cb4dccabaf90393b403f79"
server_key_iv: "bf178f7be636608156e93e57a987704a"
server_key_version: "encryptionkey@0"
I20260806 18:42:46.933960 446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "27db750cc47c45beb5690eb887b5b57a"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "73f8d8e7c4cb4dccabaf90393b403f79"
server_key_iv: "bf178f7be636608156e93e57a987704a"
server_key_version: "encryptionkey@0"
I20260806 18:42:46.937682 446 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:42:46.940173 466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:46.941536 446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:42:46.941644 446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "27db750cc47c45beb5690eb887b5b57a"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "73f8d8e7c4cb4dccabaf90393b403f79"
server_key_iv: "bf178f7be636608156e93e57a987704a"
server_key_version: "encryptionkey@0"
I20260806 18:42:46.941741 446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:46.974831 446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:46.977886 446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:46.978087 446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:46.978703 446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:46.979636 446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:46.979744 446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:46.979859 446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:46.979889 446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:46.988716 446 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:36097
I20260806 18:42:46.988745 636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:36097 every 8 connection(s)
I20260806 18:42:46.989673 446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:42:46.995987 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 446
I20260806 18:42:46.996078 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:42:46.996312 9159 external_mini_cluster.cc:1428] Setting key 59d2f2cdeee167e68185ba13116a1553
Aug 06 18:42:46 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041766, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:47.003446 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:46.991506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:50189 (local address 127.8.241.254:39103)
0806 18:42:46.991644 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:46.991648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:46.992538 (+ 890us) server_negotiation.cc:407] Connection header received
0806 18:42:46.993902 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:46.993905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:46.993955 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:46.994027 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:46.995155 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.995689 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.996879 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.997201 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.997286 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:46.999897 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:46.999900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:46.999902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:46.999925 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:47.001354 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:47.001914 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:47.001917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:47.001918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:47.001997 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:47.002330 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:47.002332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:47.002333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:47.002497 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:47.002567 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:47.003215 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 18:42:47.003318 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:42:47.004372 641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:46.991926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:50189)
0806 18:42:46.992392 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:46.992435 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:46.993695 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:46.994165 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:46.994172 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:46.994527 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:46.995014 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:46.995028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.995796 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.995799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:46.996759 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:46.996770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:46.997355 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:46.997359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:46.997564 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:46.998119 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:46.998135 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:46.999727 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:47.001468 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:47.001474 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:47.001491 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:47.001831 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:47.002072 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:47.002075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:47.002077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:47.002254 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:47.002603 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:47.002611 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:47.002955 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 18:42:47.003181 (+ 226us) client_negotiation.cc:240] Negotiation successful
0806 18:42:47.003542 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260806 18:42:47.005851 637 heartbeater.cc:347] Connected to a master server at 127.8.241.254:39103
I20260806 18:42:47.006090 637 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:47.006577 637 heartbeater.cc:511] Master 127.8.241.254:39103 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:42:47.007715 32627 ts_manager.cc:194] Registered new tserver with Master: 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097)
I20260806 18:42:47.008309 32627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:50189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:42:47.024140 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39103
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:42:47.141970 644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:42:47.142221 644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:42:47.142263 644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:42:47.146275 644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:42:47.146337 644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:42:47.146392 644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:42:47.151672 644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:42:47.152685 644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:42:47.153716 644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:42:47.163385 649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:47.163408 650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:42:47.163442 652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:42:47.163498 644 server_base.cc:1034] running on GCE node
I20260806 18:42:47.163931 644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:42:47.164618 644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:42:47.165779 644 hybrid_clock.cc:648] HybridClock initialized: now 1786041767165760 us; error 32 us; skew 500 ppm
Aug 06 18:42:47 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:42:47.168491 644 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:42:47.169409 644 webserver.cc:460] Webserver started at http://127.8.241.195:40633/ using document root <none> and password file <none>
I20260806 18:42:47.169971 644 fs_manager.cc:359] Metadata directory not provided
I20260806 18:42:47.170040 644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:42:47.170250 644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:42:47.172292 644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "6ad1e7bcc4c1428193386df77cc6b74a"
format_stamp: "Formatted at 2026-08-06 18:42:47 on dist-test-slave-k5rr"
server_key: "eaaa0be5a7a7004914fa30c7a9ade340"
server_key_iv: "2be56a2587f383ab330e194ee191ea8b"
server_key_version: "encryptionkey@0"
I20260806 18:42:47.172845 644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "6ad1e7bcc4c1428193386df77cc6b74a"
format_stamp: "Formatted at 2026-08-06 18:42:47 on dist-test-slave-k5rr"
server_key: "eaaa0be5a7a7004914fa30c7a9ade340"
server_key_iv: "2be56a2587f383ab330e194ee191ea8b"
server_key_version: "encryptionkey@0"
I20260806 18:42:47.176503 644 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:42:47.178848 662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:47.180035 644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:42:47.180157 644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "6ad1e7bcc4c1428193386df77cc6b74a"
format_stamp: "Formatted at 2026-08-06 18:42:47 on dist-test-slave-k5rr"
server_key: "eaaa0be5a7a7004914fa30c7a9ade340"
server_key_iv: "2be56a2587f383ab330e194ee191ea8b"
server_key_version: "encryptionkey@0"
I20260806 18:42:47.180256 644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:42:47.193979 644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:42:47.197160 644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:42:47.197342 644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:42:47.197944 644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:42:47.198864 644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:42:47.198913 644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:47.198968 644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:42:47.198994 644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:42:47.207331 644 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:37471
I20260806 18:42:47.207387 832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:37471 every 8 connection(s)
I20260806 18:42:47.208254 644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:42:47.211721 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 644
I20260806 18:42:47.211855 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:42:47.212179 9159 external_mini_cluster.cc:1428] Setting key c08021cf8d8d2a633ed01aed8387c96a
Aug 06 18:42:47 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:42:47.221915 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.210139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:44211 (local address 127.8.241.254:39103)
0806 18:42:47.210318 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:47.210323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:47.211189 (+ 866us) server_negotiation.cc:407] Connection header received
0806 18:42:47.212799 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:47.212803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:47.212859 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:47.212935 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:47.214404 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.214931 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.216035 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.216293 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.216370 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.218732 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:47.218736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:47.218738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:47.218762 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:47.220077 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:47.220531 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:47.220533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:47.220534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:47.220606 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:47.220857 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:47.220859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:47.220860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:47.221131 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:47.221212 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:47.221684 (+ 472us) server_negotiation.cc:299] Negotiation successful
0806 18:42:47.221789 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 18:42:47.222798 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.210569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:44211)
0806 18:42:47.211038 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:47.211082 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:47.212604 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:47.213148 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:47.213155 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:47.213605 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:42:47.214275 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.214295 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.215047 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.215049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.215934 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.215944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.216407 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.216410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.216565 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.217103 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:42:47.217120 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:42:47.218582 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:42:47.220183 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:47.220188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:47.220200 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:47.220459 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:47.220669 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:42:47.220672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:42:47.220673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:42:47.220794 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:42:47.221231 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:42:47.221237 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:42:47.221482 (+ 245us) client_negotiation.cc:769] Sending connection context
0806 18:42:47.221648 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 18:42:47.221915 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":125,"threads_started":1}
I20260806 18:42:47.223896 833 heartbeater.cc:347] Connected to a master server at 127.8.241.254:39103
I20260806 18:42:47.224123 833 heartbeater.cc:464] Registering TS with master...
I20260806 18:42:47.224694 833 heartbeater.cc:511] Master 127.8.241.254:39103 requested a full tablet report, sending...
I20260806 18:42:47.225829 32627 ts_manager.cc:194] Registered new tserver with Master: 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471)
I20260806 18:42:47.226390 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:42:47.226589 32627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:44211
I20260806 18:42:47.235136 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.228558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.8.241.254:39103)
0806 18:42:47.228685 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:47.228689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:47.228821 (+ 132us) server_negotiation.cc:407] Connection header received
0806 18:42:47.228935 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:47.228938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:47.228983 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:47.229095 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:42:47.229812 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.230332 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.231160 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.231368 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.231467 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.232663 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:42:47.232667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:42:47.232669 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:42:47.232684 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:42:47.233834 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:47.234194 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:47.234196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:47.234197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:47.234260 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:42:47.234452 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:42:47.234454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:42:47.234455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:42:47.234649 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:42:47.234751 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:47.234907 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 18:42:47.234997 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:42:47.237690 32627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:42:47.240648 32627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:42:47.255046 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.248984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:40419 (local address 127.0.0.1:36328)
0806 18:42:47.249736 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:47.249749 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:47.249845 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:47.250851 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:47.250854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:47.250873 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:47.250981 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.250987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.252089 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.252092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.253544 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.253556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.254544 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.254548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.254639 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.254667 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:42:47.254717 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 18:42:47.254887 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":56,"thread_start_us":87,"threads_started":1}
I20260806 18:42:47.255034 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.249581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.8.241.194:36097)
0806 18:42:47.250213 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:47.250224 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:47.250248 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:42:47.250329 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:47.250335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:47.250531 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:47.250693 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:47.251068 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.252236 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.253687 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.254363 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.254470 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.254628 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:47.254769 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 18:42:47.254863 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":142,"threads_started":1}
I20260806 18:42:47.255296 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.248859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36097 (local address 127.0.0.1:59722)
0806 18:42:47.249224 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:47.249269 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:47.249387 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:47.250758 (+ 1371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:47.250765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:47.250788 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:47.250924 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.250932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.252385 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.252389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.253574 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.253581 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.254476 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.254480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.254595 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.254634 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:42:47.254738 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 18:42:47.254887 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":88,"thread_start_us":87,"threads_started":1}
I20260806 18:42:47.255541 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.250084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.8.241.193:40419)
0806 18:42:47.250380 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:47.250397 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:47.250422 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:42:47.250504 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:47.250508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:47.250701 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:47.250851 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:47.251079 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.251973 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.253693 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.254387 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.255010 (+ 623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.255134 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:47.255191 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 18:42:47.255290 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260806 18:42:47.256325 860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.249226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37471 (local address 127.0.0.1:43904)
0806 18:42:47.249783 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:47.249794 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:47.249857 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:47.251482 (+ 1625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:47.251485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:47.251504 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:47.251596 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.251601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.252802 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.252805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.253894 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.253906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.255909 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.255913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.256019 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.256044 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:42:47.256112 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:42:47.256176 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260806 18:42:47.257067 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.249703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43904 (local address 127.8.241.195:37471)
0806 18:42:47.251039 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:47.251052 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:47.251072 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:42:47.251142 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:47.251147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:47.251324 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:47.251481 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:47.251704 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.252657 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.254038 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.254803 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.254897 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.255087 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:47.256784 (+ 1697us) server_negotiation.cc:299] Negotiation successful
0806 18:42:47.256897 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1220,"thread_start_us":140,"threads_started":1}
I20260806 18:42:47.258435 343 tablet_service.cc:1467] Processing CreateTablet for tablet 5c5289c5980844fe97304a6a77f996f5 (DEFAULT_TABLE table=test-table [id=66be04a07fc6488db5867996fe87cde0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:47.258463 530 tablet_service.cc:1467] Processing CreateTablet for tablet 5c5289c5980844fe97304a6a77f996f5 (DEFAULT_TABLE table=test-table [id=66be04a07fc6488db5867996fe87cde0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:47.259452 343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c5289c5980844fe97304a6a77f996f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:47.259531 530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c5289c5980844fe97304a6a77f996f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:47.259810 732 tablet_service.cc:1467] Processing CreateTablet for tablet 5c5289c5980844fe97304a6a77f996f5 (DEFAULT_TABLE table=test-table [id=66be04a07fc6488db5867996fe87cde0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:42:47.260819 732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c5289c5980844fe97304a6a77f996f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:42:47.265834 864 tablet_bootstrap.cc:492] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Bootstrap starting.
I20260806 18:42:47.267184 865 tablet_bootstrap.cc:492] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: Bootstrap starting.
I20260806 18:42:47.268464 864 tablet_bootstrap.cc:654] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:47.269297 866 tablet_bootstrap.cc:492] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: Bootstrap starting.
I20260806 18:42:47.269923 865 tablet_bootstrap.cc:654] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:47.269964 864 log.cc:824] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:47.270782 865 log.cc:824] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:47.271191 866 tablet_bootstrap.cc:654] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: Neither blocks nor log segments found. Creating new log.
I20260806 18:42:47.272157 864 tablet_bootstrap.cc:492] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: No bootstrap required, opened a new log
I20260806 18:42:47.272317 866 log.cc:824] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:42:47.272370 864 ts_tablet_manager.cc:1397] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260806 18:42:47.272850 865 tablet_bootstrap.cc:492] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: No bootstrap required, opened a new log
I20260806 18:42:47.273116 865 ts_tablet_manager.cc:1397] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:42:47.274017 866 tablet_bootstrap.cc:492] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: No bootstrap required, opened a new log
I20260806 18:42:47.274192 866 ts_tablet_manager.cc:1397] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:42:47.276276 864 raft_consensus.cc:348] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:42:47.276562 864 raft_consensus.cc:374] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:47.276647 864 raft_consensus.cc:729] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181bfe4e15e145b39095c438caccc52b, State: Initialized, Role: FOLLOWER
I20260806 18:42:47.276772 865 raft_consensus.cc:348] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:42:47.277040 865 raft_consensus.cc:374] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:47.276890 866 raft_consensus.cc:348] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:42:47.277132 865 raft_consensus.cc:729] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27db750cc47c45beb5690eb887b5b57a, State: Initialized, Role: FOLLOWER
I20260806 18:42:47.277216 866 raft_consensus.cc:374] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:42:47.277154 864 consensus_queue.cc:260] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:42:47.277280 866 raft_consensus.cc:729] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1e7bcc4c1428193386df77cc6b74a, State: Initialized, Role: FOLLOWER
I20260806 18:42:47.277585 865 consensus_queue.cc:260] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:42:47.277722 866 consensus_queue.cc:260] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:42:47.278189 864 ts_tablet_manager.cc:1428] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:42:47.278509 865 ts_tablet_manager.cc:1428] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:42:47.278568 441 heartbeater.cc:503] Master 127.8.241.254:39103 was elected leader, sending a full tablet report...
I20260806 18:42:47.278657 833 heartbeater.cc:503] Master 127.8.241.254:39103 was elected leader, sending a full tablet report...
I20260806 18:42:47.278678 866 ts_tablet_manager.cc:1428] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 18:42:47.279140 637 heartbeater.cc:503] Master 127.8.241.254:39103 was elected leader, sending a full tablet report...
I20260806 18:42:47.325433 871 raft_consensus.cc:482] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:42:47.325641 871 raft_consensus.cc:504] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:42:47.326826 871 leader_election.cc:288] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097), 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471)
I20260806 18:42:47.330890 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.327248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:41453 (local address 127.8.241.194:36097)
0806 18:42:47.327471 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:47.327476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:47.327513 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:42:47.327675 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:47.327677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:47.327722 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:47.327783 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:47.328155 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.328644 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.329901 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.330321 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.330440 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.330494 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:47.330679 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:42:47.330759 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260806 18:42:47.330907 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.327056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36097 (local address 127.8.241.193:41453)
0806 18:42:47.327437 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:47.327452 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:47.327569 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:47.327802 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:47.327804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:47.327821 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:47.327969 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.327975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.328754 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.328758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.329783 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.329794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.330441 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.330445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.330536 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.330565 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:42:47.330642 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:42:47.330714 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":42,"spinlock_wait_cycles":11648,"thread_start_us":171,"threads_started":1}
I20260806 18:42:47.330927 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.327230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:40771 (local address 127.8.241.195:37471)
0806 18:42:47.327399 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:47.327404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:47.327601 (+ 197us) server_negotiation.cc:407] Connection header received
0806 18:42:47.327711 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:47.327714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:47.327754 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:47.327814 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:42:47.328123 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.328626 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.329903 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.330319 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.330426 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.330504 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:47.330743 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 18:42:47.330824 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 18:42:47.330950 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.327072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37471 (local address 127.8.241.193:40771)
0806 18:42:47.327510 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:47.327521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:42:47.327624 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:42:47.327833 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:42:47.327836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:42:47.327851 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:42:47.327983 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.327988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.328734 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.328737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.329785 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:42:47.329796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.330437 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.330449 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:42:47.330570 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.330594 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:42:47.330699 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 18:42:47.330763 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":91,"thread_start_us":159,"threads_started":1}
I20260806 18:42:47.331640 772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c5289c5980844fe97304a6a77f996f5" candidate_uuid: "181bfe4e15e145b39095c438caccc52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" is_pre_election: true
I20260806 18:42:47.331965 772 raft_consensus.cc:2454] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 181bfe4e15e145b39095c438caccc52b in term 0.
I20260806 18:42:47.332474 32746 leader_election.cc:302] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181bfe4e15e145b39095c438caccc52b, 6ad1e7bcc4c1428193386df77cc6b74a; no voters:
I20260806 18:42:47.332525 567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c5289c5980844fe97304a6a77f996f5" candidate_uuid: "181bfe4e15e145b39095c438caccc52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27db750cc47c45beb5690eb887b5b57a" is_pre_election: true
I20260806 18:42:47.332790 871 raft_consensus.cc:2781] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:42:47.332834 567 raft_consensus.cc:2454] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 181bfe4e15e145b39095c438caccc52b in term 0.
I20260806 18:42:47.332876 871 raft_consensus.cc:482] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:42:47.332929 871 raft_consensus.cc:3037] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:47.334061 871 raft_consensus.cc:504] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:42:47.334398 871 leader_election.cc:288] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 1 election: Requested vote from peers 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097), 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471)
I20260806 18:42:47.334705 567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c5289c5980844fe97304a6a77f996f5" candidate_uuid: "181bfe4e15e145b39095c438caccc52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27db750cc47c45beb5690eb887b5b57a"
I20260806 18:42:47.334753 772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c5289c5980844fe97304a6a77f996f5" candidate_uuid: "181bfe4e15e145b39095c438caccc52b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad1e7bcc4c1428193386df77cc6b74a"
I20260806 18:42:47.334830 567 raft_consensus.cc:3037] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:47.334862 772 raft_consensus.cc:3037] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:42:47.335824 567 raft_consensus.cc:2454] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 181bfe4e15e145b39095c438caccc52b in term 1.
I20260806 18:42:47.335896 772 raft_consensus.cc:2454] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 181bfe4e15e145b39095c438caccc52b in term 1.
I20260806 18:42:47.336124 32749 leader_election.cc:302] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181bfe4e15e145b39095c438caccc52b, 27db750cc47c45beb5690eb887b5b57a; no voters:
I20260806 18:42:47.336310 871 raft_consensus.cc:2781] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:42:47.336534 871 raft_consensus.cc:686] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 1 LEADER]: Becoming Leader. State: Replica: 181bfe4e15e145b39095c438caccc52b, State: Running, Role: LEADER
I20260806 18:42:47.336881 871 consensus_queue.cc:237] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:42:47.340121 32623 catalog_manager.cc:5495] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b reported cstate change: term changed from 0 to 1, leader changed from <none> to 181bfe4e15e145b39095c438caccc52b (127.8.241.193). New cstate: current_term: 1 leader_uuid: "181bfe4e15e145b39095c438caccc52b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } health_report { overall_health: UNKNOWN } } }
I20260806 18:42:47.351881 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.348763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.8.241.193:40419)
0806 18:42:47.348894 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:42:47.348898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:42:47.348912 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:42:47.349022 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:42:47.349025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:42:47.349086 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:42:47.349167 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:42:47.349368 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.349874 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.350640 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:42:47.350857 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:42:47.350938 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:42:47.351168 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:42:47.351599 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:42:47.351691 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:42:47.351720 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 18:42:47.351771 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:42:47.357851 567 raft_consensus.cc:1261] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [term 1 FOLLOWER]: Refusing update from remote peer 181bfe4e15e145b39095c438caccc52b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:47.357851 772 raft_consensus.cc:1261] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [term 1 FOLLOWER]: Refusing update from remote peer 181bfe4e15e145b39095c438caccc52b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:42:47.358577 883 consensus_queue.cc:1035] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:47.358758 871 consensus_queue.cc:1035] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [LEADER]: Connected to new peer: Peer: permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:42:47.375867 32625 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:42:47.376081 32625 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:42:47.378973 32625 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Sending DeleteTablet for 3 replicas of tablet 5c5289c5980844fe97304a6a77f996f5
I20260806 18:42:47.379858 343 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c5289c5980844fe97304a6a77f996f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:47 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:36328
I20260806 18:42:47.379874 530 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c5289c5980844fe97304a6a77f996f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:47 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:59722
I20260806 18:42:47.379877 732 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c5289c5980844fe97304a6a77f996f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:47 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:43904
I20260806 18:42:47.380420 902 tablet_replica.cc:330] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: stopping tablet replica
I20260806 18:42:47.380502 903 tablet_replica.cc:330] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: stopping tablet replica
I20260806 18:42:47.380734 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 32723
I20260806 18:42:47.380831 902 raft_consensus.cc:2229] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:42:47.380829 903 raft_consensus.cc:2229] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:42:47.388362 903 raft_consensus.cc:2258] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:42:47.389909 902 raft_consensus.cc:2258] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:42:47.390975 903 ts_tablet_manager.cc:1905] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:42:47.391598 902 ts_tablet_manager.cc:1905] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:42:47.393991 903 ts_tablet_manager.cc:1918] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:42:47.394110 903 log.cc:1196] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/5c5289c5980844fe97304a6a77f996f5
I20260806 18:42:47.394284 902 ts_tablet_manager.cc:1918] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:42:47.394383 902 log.cc:1196] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/5c5289c5980844fe97304a6a77f996f5
I20260806 18:42:47.394538 903 ts_tablet_manager.cc:1939] T 5c5289c5980844fe97304a6a77f996f5 P 6ad1e7bcc4c1428193386df77cc6b74a: Deleting consensus metadata
I20260806 18:42:47.394745 902 ts_tablet_manager.cc:1939] T 5c5289c5980844fe97304a6a77f996f5 P 27db750cc47c45beb5690eb887b5b57a: Deleting consensus metadata
I20260806 18:42:47.396046 32608 catalog_manager.cc:4841] TS 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097): tablet 5c5289c5980844fe97304a6a77f996f5 (table test-table [id=66be04a07fc6488db5867996fe87cde0]) successfully deleted
I20260806 18:42:47.396046 32605 catalog_manager.cc:4841] TS 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471): tablet 5c5289c5980844fe97304a6a77f996f5 (table test-table [id=66be04a07fc6488db5867996fe87cde0]) successfully deleted
W20260806 18:42:47.399648 32607 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:40419 (error 108)
I20260806 18:42:47.400508 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 446
I20260806 18:42:47.400723 860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:42:47.400434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:40419 (local address 127.0.0.1:36350)
0806 18:42:47.400567 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:42:47.400638 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:40419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 18:42:47.407689 32607 catalog_manager.cc:4584] TS 181bfe4e15e145b39095c438caccc52b (127.8.241.193:40419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c5289c5980844fe97304a6a77f996f5: Network error: Client connection negotiation failed: client connection to 127.8.241.193:40419: connect: Connection refused (error 111)
I20260806 18:42:47.415679 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 644
I20260806 18:42:47.439189 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 32580
I20260806 18:43:07.451122 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39103
--webserver_interface=127.8.241.254
--webserver_port=33977
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:07.573771 906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:07.574082 906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:07.574150 906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:07.578322 906 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:43:07.578380 906 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:43:07.578406 906 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:43:07.578429 906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:07.578469 906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:07.578508 906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:07.578542 906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:07.578576 906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:07.584003 906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=33977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:07.585201 906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:07.586292 906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:07.600618 914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:07.600812 906 server_base.cc:1034] running on GCE node
W20260806 18:43:07.600626 916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:07.600653 913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:07.601541 906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:07.602689 906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:07.603870 906 hybrid_clock.cc:648] HybridClock initialized: now 1786041787603848 us; error 32 us; skew 500 ppm
Aug 06 18:43:07 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041787, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:07.606770 906 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:43:07.607717 906 webserver.cc:460] Webserver started at http://127.8.241.254:33977/ using document root <none> and password file <none>
I20260806 18:43:07.608304 906 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:07.608386 906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:07.612311 906 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:07.614516 926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:07.615746 906 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:43:07.615854 906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "a9aafb8d917a40218b9f38a792af6d31"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "62b20c342a74bde5d35c7b5d9151e64e"
server_key_iv: "45a939d8ef29cb01dac82f8d4cf5fb92"
server_key_version: "encryptionkey@0"
I20260806 18:43:07.616464 906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:07.649387 906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:07.652274 906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:07.652436 906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:07.660156 906 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:39103
I20260806 18:43:07.660446 997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:39103 every 8 connection(s)
I20260806 18:43:07.661338 906 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:07.665912 999 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:43:07.667647 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Bootstrap starting.
I20260806 18:43:07.669512 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 906
I20260806 18:43:07.669912 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:40419
--local_ip_for_outbound_sockets=127.8.241.193
--tserver_master_addrs=127.8.241.254:39103
--webserver_port=40949
--webserver_interface=127.8.241.193
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:43:07.685858 999 log.cc:824] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:07.694986 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:43:07.695330 999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Bootstrap complete.
I20260806 18:43:07.699373 999 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:43:07.699779 999 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9aafb8d917a40218b9f38a792af6d31, State: Initialized, Role: FOLLOWER
I20260806 18:43:07.700277 999 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:43:07.700407 999 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:07.700481 999 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:07.700577 999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:43:07.702517 999 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:43:07.702845 999 leader_election.cc:302] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9aafb8d917a40218b9f38a792af6d31; no voters:
I20260806 18:43:07.703142 999 leader_election.cc:288] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:43:07.703243 1006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:43:07.703457 1006 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 2 LEADER]: Becoming Leader. State: Replica: a9aafb8d917a40218b9f38a792af6d31, State: Running, Role: LEADER
I20260806 18:43:07.703821 1006 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:43:07.704432 999 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:07.705338 1008 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9aafb8d917a40218b9f38a792af6d31. Latest consensus state: current_term: 2 leader_uuid: "a9aafb8d917a40218b9f38a792af6d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } } }
I20260806 18:43:07.705477 1008 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:07.706344 1007 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a9aafb8d917a40218b9f38a792af6d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } } }
I20260806 18:43:07.706446 1007 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:07.706663 1016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:07.710618 1016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:07.711130 1016 catalog_manager.cc:1259] Loaded cluster ID: 4a43eb72728d421aaa81913a02fedd50
I20260806 18:43:07.711181 1016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:07.713073 1016 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:07.713675 1016 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Loaded TSK: 0
I20260806 18:43:07.725471 1016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Generated new TSK 1
I20260806 18:43:07.725728 1016 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:43:07.796089 1001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:07.796350 1001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:07.796409 1001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:07.800495 1001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:07.800554 1001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:07.800609 1001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:43:07.805805 1001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:40419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=40949
--tserver_master_addrs=127.8.241.254:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:07.806887 1001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:07.807910 1001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:07.817605 1040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:07.817636 1041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:07.817848 1001 server_base.cc:1034] running on GCE node
W20260806 18:43:07.818162 1044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:07.818652 1001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:07.819573 1001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:07.820792 1001 hybrid_clock.cc:648] HybridClock initialized: now 1786041787820774 us; error 31 us; skew 500 ppm
Aug 06 18:43:07 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041787, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:07.830950 1001 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:43:07.832279 1001 webserver.cc:460] Webserver started at http://127.8.241.193:40949/ using document root <none> and password file <none>
I20260806 18:43:07.833088 1001 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:07.833191 1001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:07.837292 1001 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:43:07.839455 1052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:07.840739 1001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:07.840842 1001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "181bfe4e15e145b39095c438caccc52b"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "3b692be522385de50bdaacca05b52d9b"
server_key_iv: "905e38dfedd112d9318b7fae9c6cfc13"
server_key_version: "encryptionkey@0"
I20260806 18:43:07.841454 1001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:07.854986 1001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:07.857841 1001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:07.858014 1001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:07.858623 1001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:07.859710 1063 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:43:07.863878 1001 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:43:07.863935 1001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 18:43:07.863999 1001 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:43:07.865720 1001 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:43:07.865765 1001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260806 18:43:07.866044 1063 tablet_bootstrap.cc:492] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Bootstrap starting.
I20260806 18:43:07.876260 1001 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:40419
I20260806 18:43:07.876356 1229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:40419 every 8 connection(s)
I20260806 18:43:07.877316 1001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:43:07.877799 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 1001
I20260806 18:43:07.878288 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:36097
--local_ip_for_outbound_sockets=127.8.241.194
--tserver_master_addrs=127.8.241.254:39103
--webserver_port=35117
--webserver_interface=127.8.241.194
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:43:07.887099 1063 log.cc:824] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:43:07 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041787, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:07.895428 1063 tablet_bootstrap.cc:492] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:43:07.895931 1063 tablet_bootstrap.cc:492] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Bootstrap complete.
I20260806 18:43:07.896488 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:07.879424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:48011 (local address 127.8.241.254:39103)
0806 18:43:07.879944 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:07.879964 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:07.880407 (+ 443us) server_negotiation.cc:407] Connection header received
0806 18:43:07.881551 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:07.881562 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:07.881849 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:07.882159 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:07.883760 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:07.885350 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:07.886956 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:07.887701 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:07.887881 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:07.890875 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:07.890882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:07.890899 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:07.890933 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:07.892902 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:07.893523 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:07.893529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:07.893535 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:07.893683 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:07.894114 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:07.894117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:07.894118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:07.894563 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:07.894693 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:07.895316 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 18:43:07.895623 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":115,"threads_started":1}
I20260806 18:43:07.896508 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:07.879445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:48011)
0806 18:43:07.880236 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:07.880290 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:07.881144 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:07.882256 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:07.882266 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:07.882778 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:07.883530 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:07.883553 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:07.885529 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:07.885533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:07.886816 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:07.886828 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:07.887866 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:07.887871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:07.888055 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:07.888679 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:07.888702 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:07.890647 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:07.893053 (+ 2406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:07.893060 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:07.893074 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:07.893401 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:07.893820 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:07.893824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:07.893826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:07.894001 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:07.894698 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:07.894704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:07.894985 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 18:43:07.895181 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 18:43:07.895497 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":135,"threads_started":1}
I20260806 18:43:07.897490 1063 ts_tablet_manager.cc:1397] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.012s
I20260806 18:43:07.899369 1230 heartbeater.cc:347] Connected to a master server at 127.8.241.254:39103
I20260806 18:43:07.899613 1230 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:07.900162 1230 heartbeater.cc:511] Master 127.8.241.254:39103 requested a full tablet report, sending...
I20260806 18:43:07.900089 1063 raft_consensus.cc:348] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:43:07.900583 1063 raft_consensus.cc:729] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181bfe4e15e145b39095c438caccc52b, State: Initialized, Role: FOLLOWER
I20260806 18:43:07.901072 1063 consensus_queue.cc:260] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } }
I20260806 18:43:07.902926 1063 ts_tablet_manager.cc:1428] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:43:07.903565 950 ts_manager.cc:194] Registered new tserver with Master: 181bfe4e15e145b39095c438caccc52b (127.8.241.193:40419)
I20260806 18:43:07.906934 950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:48011
I20260806 18:43:07.909379 1230 heartbeater.cc:503] Master 127.8.241.254:39103 was elected leader, sending a full tablet report...
Aug 06 18:43:07 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041787, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.193@KRBTEST.COM
I20260806 18:43:07.917420 1246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:07.906442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:40419 (local address 127.0.0.1:44510)
0806 18:43:07.907278 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:07.907322 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:07.907456 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:07.908168 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:07.908174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:07.908593 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:07.908723 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:07.908732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:07.910117 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:07.910121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:07.911052 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:07.911062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:07.911780 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:07.911783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:07.911878 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:07.912381 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:07.912393 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:07.913912 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:07.915701 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:07.915711 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:07.915717 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:07.915933 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:07.916213 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:07.916216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:07.916218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:07.916307 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:07.916737 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:07.916743 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:07.916866 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 18:43:07.916982 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 18:43:07.917241 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":131,"threads_started":1}
I20260806 18:43:07.917420 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:07.906520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44510 (local address 127.8.241.193:40419)
0806 18:43:07.907717 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:07.907728 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:07.907749 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:43:07.907816 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:07.907821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:07.908007 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:07.908135 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:07.908858 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:07.909987 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:07.911157 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:07.911654 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:07.911758 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:07.914136 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:07.914146 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:07.914159 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:07.914182 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:07.915594 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:07.916040 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:07.916047 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:07.916048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:07.916126 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:07.916392 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:07.916396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:07.916398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:07.916620 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:07.916740 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:07.917121 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 18:43:07.917251 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1103,"thread_start_us":91,"threads_started":1}
I20260806 18:43:07.918628 1123 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c5289c5980844fe97304a6a77f996f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:42:47 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:44510
I20260806 18:43:07.919464 1248 tablet_replica.cc:330] stopping tablet replica
I20260806 18:43:07.919746 1248 raft_consensus.cc:2229] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:07.919996 1248 raft_consensus.cc:2258] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:07.920912 1248 ts_tablet_manager.cc:1905] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:07.924511 1248 ts_tablet_manager.cc:1918] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:07.924594 1248 log.cc:1196] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal/wals/5c5289c5980844fe97304a6a77f996f5
I20260806 18:43:07.924835 1248 ts_tablet_manager.cc:1939] T 5c5289c5980844fe97304a6a77f996f5 P 181bfe4e15e145b39095c438caccc52b: Deleting consensus metadata
I20260806 18:43:07.926470 929 catalog_manager.cc:4841] TS 181bfe4e15e145b39095c438caccc52b (127.8.241.193:40419): tablet 5c5289c5980844fe97304a6a77f996f5 (table test-table [id=66be04a07fc6488db5867996fe87cde0]) successfully deleted
W20260806 18:43:08.000031 1239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:08.000278 1239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:08.000337 1239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:08.004310 1239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:08.004374 1239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:08.004456 1239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:43:08.009812 1239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:36097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=35117
--tserver_master_addrs=127.8.241.254:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:08.010842 1239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:08.011868 1239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:08.021766 1257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:08.021775 1254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:08.021852 1239 server_base.cc:1034] running on GCE node
W20260806 18:43:08.021786 1255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:08.022440 1239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:08.023034 1239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:08.024191 1239 hybrid_clock.cc:648] HybridClock initialized: now 1786041788024158 us; error 43 us; skew 500 ppm
Aug 06 18:43:08 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041788, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:08.026886 1239 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:43:08.027777 1239 webserver.cc:460] Webserver started at http://127.8.241.194:35117/ using document root <none> and password file <none>
I20260806 18:43:08.028386 1239 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:08.028476 1239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:08.032347 1239 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:08.034462 1271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:08.035724 1239 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:43:08.035836 1239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "27db750cc47c45beb5690eb887b5b57a"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "73f8d8e7c4cb4dccabaf90393b403f79"
server_key_iv: "bf178f7be636608156e93e57a987704a"
server_key_version: "encryptionkey@0"
I20260806 18:43:08.036371 1239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:08.077984 1239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:08.081313 1239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:08.081517 1239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:08.082155 1239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:08.083026 1239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:08.083076 1239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:08.083133 1239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:08.083161 1239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:08.094300 1239 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:36097
I20260806 18:43:08.094339 1445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:36097 every 8 connection(s)
I20260806 18:43:08.095301 1239 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:43:08.096853 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 1239
I20260806 18:43:08.097368 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:37471
--local_ip_for_outbound_sockets=127.8.241.195
--tserver_master_addrs=127.8.241.254:39103
--webserver_port=40633
--webserver_interface=127.8.241.195
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:43:08 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041788, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:08.110198 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.097185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:36347 (local address 127.8.241.254:39103)
0806 18:43:08.097336 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.097341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.098444 (+ 1103us) server_negotiation.cc:407] Connection header received
0806 18:43:08.099853 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.099858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.099927 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.100004 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:08.101318 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.101852 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.103532 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.103768 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.103849 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.106788 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:08.106791 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:08.106793 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:08.106818 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:08.108162 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:08.108670 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:08.108673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:08.108674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:08.108751 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:08.109101 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:08.109103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:08.109104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:08.109285 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:08.109371 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:08.109883 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 18:43:08.110032 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:43:08.110986 1448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.097727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:36347)
0806 18:43:08.098278 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:08.098325 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:08.099619 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:08.100194 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:08.100202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:08.100578 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:08.101151 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.101173 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.101991 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.101994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.103405 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.103419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.103921 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.103926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.104133 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.104746 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:08.104768 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:08.106610 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:08.108274 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:08.108285 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:08.108300 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:08.108565 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:08.108834 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:08.108837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:08.108839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:08.108981 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:08.109389 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:08.109396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:08.109646 (+ 250us) client_negotiation.cc:769] Sending connection context
0806 18:43:08.109808 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 18:43:08.110133 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":153,"threads_started":1}
I20260806 18:43:08.112250 1446 heartbeater.cc:347] Connected to a master server at 127.8.241.254:39103
I20260806 18:43:08.112481 1446 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:08.112982 1446 heartbeater.cc:511] Master 127.8.241.254:39103 requested a full tablet report, sending...
I20260806 18:43:08.114166 950 ts_manager.cc:194] Registered new tserver with Master: 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097)
I20260806 18:43:08.114929 950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:36347
W20260806 18:43:08.214699 1449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:08.214951 1449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:08.214993 1449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:08.218920 1449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:08.218968 1449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:08.219024 1449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:43:08.224252 1449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:37471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=40633
--tserver_master_addrs=127.8.241.254:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:08.225404 1449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:08.226405 1449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:43:08.235869 1449 server_base.cc:1034] running on GCE node
W20260806 18:43:08.235832 1459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:08.235827 1457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:08.235836 1456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:08.236452 1449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:08.237087 1449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:08.238248 1449 hybrid_clock.cc:648] HybridClock initialized: now 1786041788238223 us; error 34 us; skew 500 ppm
Aug 06 18:43:08 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041788, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:08.241176 1449 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:43:08.242043 1449 webserver.cc:460] Webserver started at http://127.8.241.195:40633/ using document root <none> and password file <none>
I20260806 18:43:08.242587 1449 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:08.242667 1449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:08.246510 1449 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:43:08.248481 1468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:08.249711 1449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:08.249819 1449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "6ad1e7bcc4c1428193386df77cc6b74a"
format_stamp: "Formatted at 2026-08-06 18:42:47 on dist-test-slave-k5rr"
server_key: "eaaa0be5a7a7004914fa30c7a9ade340"
server_key_iv: "2be56a2587f383ab330e194ee191ea8b"
server_key_version: "encryptionkey@0"
I20260806 18:43:08.250342 1449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:08.263114 1449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:08.265781 1449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:08.265947 1449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:08.266503 1449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:08.267295 1449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:08.267341 1449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:08.267393 1449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:08.267422 1449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:08.275923 1449 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:37471
I20260806 18:43:08.275949 1638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:37471 every 8 connection(s)
I20260806 18:43:08.276873 1449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:43:08.285530 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 1449
Aug 06 18:43:08 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041788, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:08.289716 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.278628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:57013 (local address 127.8.241.254:39103)
0806 18:43:08.278751 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.278756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.279554 (+ 798us) server_negotiation.cc:407] Connection header received
0806 18:43:08.280834 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.280837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.280888 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.280960 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:08.282058 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.282580 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.283651 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.283887 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.283970 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.286250 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:08.286254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:08.286256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:08.286280 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:08.287810 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:08.288264 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:08.288268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:08.288270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:08.288348 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:08.288649 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:08.288652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:08.288654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:08.288829 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:08.288913 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:08.289436 (+ 523us) server_negotiation.cc:299] Negotiation successful
0806 18:43:08.289555 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:43:08.290522 1643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.278965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:57013)
0806 18:43:08.279397 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:08.279438 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:08.280625 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:08.281140 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:08.281146 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:08.281485 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:08.281941 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.281954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.282674 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.282677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.283525 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.283533 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.284002 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.284004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.284136 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.284613 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:08.284628 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:08.286040 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:08.287908 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:08.287915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:08.287929 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:08.288172 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:08.288430 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:08.288433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:08.288435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:08.288564 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:08.288928 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:08.288934 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:08.289209 (+ 275us) client_negotiation.cc:769] Sending connection context
0806 18:43:08.289376 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 18:43:08.289671 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":117,"threads_started":1}
I20260806 18:43:08.291736 1640 heartbeater.cc:347] Connected to a master server at 127.8.241.254:39103
I20260806 18:43:08.291972 1640 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:08.292482 1640 heartbeater.cc:511] Master 127.8.241.254:39103 requested a full tablet report, sending...
I20260806 18:43:08.293634 950 ts_manager.cc:194] Registered new tserver with Master: 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471)
I20260806 18:43:08.293802 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.286229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.8.241.254:39103)
0806 18:43:08.286468 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.286472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.286603 (+ 131us) server_negotiation.cc:407] Connection header received
0806 18:43:08.286697 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.286700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.286749 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.286832 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:08.287707 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.288217 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.289093 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.289333 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.289418 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.290753 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:08.290755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:08.290757 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:08.290780 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:08.292188 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:08.292647 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:08.292650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:08.292652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:08.292726 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:08.292988 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:08.292990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:08.292991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:08.293169 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:08.293350 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:08.293498 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 18:43:08.293588 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":64,"threads_started":1}
I20260806 18:43:08.294230 950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:57013
I20260806 18:43:08.295989 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:43:08.299324 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.296558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.8.241.254:39103)
0806 18:43:08.296658 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.296661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.296807 (+ 146us) server_negotiation.cc:407] Connection header received
0806 18:43:08.296915 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.296917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.296995 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.297107 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:43:08.297260 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.297709 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.298485 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.298686 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.298749 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.298954 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:43:08.299065 (+ 111us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 18:43:08.299244 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53232: token expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:43:08.299373 1646 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53232: token expired
W20260806 18:43:08.299726 836 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.241.254:39103)
I20260806 18:43:08.305709 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.300030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.8.241.254:39103)
0806 18:43:08.300122 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.300125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.300149 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:43:08.300239 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.300242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.300273 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.300330 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:08.300805 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.301287 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.301983 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.302163 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.302222 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.303311 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:08.303314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:08.303316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:08.303335 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:08.304416 (+ 1081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:08.304779 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:08.304782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:08.304783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:08.304850 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:08.305079 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:08.305081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:08.305082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:08.305227 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:08.305320 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:08.305478 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 18:43:08.305612 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 18:43:08.307847 950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:43:08.310969 950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:43:08.318377 1123 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1428eb193341719ab670859093b3be (DEFAULT_TABLE table=test-table [id=38f5fae8bf8647c4855921a0f7bdba14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:08.319069 1123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1428eb193341719ab670859093b3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:08.322062 1063 tablet_bootstrap.cc:492] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Bootstrap starting.
I20260806 18:43:08.322263 1246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.317265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36097 (local address 127.0.0.1:43800)
0806 18:43:08.317364 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:08.317379 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:08.317452 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:08.318481 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:08.318484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:08.318501 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:08.318590 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.318595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.319759 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.319762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.320710 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.320721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.321846 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.321850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.321960 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.322012 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 18:43:08.322067 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:43:08.322145 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 18:43:08.322839 1063 tablet_bootstrap.cc:654] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:08.322885 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.317593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.8.241.194:36097)
0806 18:43:08.318061 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.318072 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.318095 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:43:08.318175 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.318181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.318352 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.318485 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:08.318684 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.319656 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.320842 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.321685 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.322510 (+ 825us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.322643 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:08.322685 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 18:43:08.322756 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":131,"threads_started":1}
I20260806 18:43:08.324251 1063 tablet_bootstrap.cc:492] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: No bootstrap required, opened a new log
I20260806 18:43:08.324360 1063 ts_tablet_manager.cc:1397] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:43:08.324476 1652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.317569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37471 (local address 127.0.0.1:60828)
0806 18:43:08.318004 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:08.318016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:08.318098 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:08.318919 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:08.318922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:08.318941 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:08.319025 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.319032 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.320358 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.320362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.321296 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.321306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.324080 (+ 2774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.324084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.324185 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.324212 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:43:08.324292 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:43:08.324360 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":115,"threads_started":1}
I20260806 18:43:08.324577 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.317853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60828 (local address 127.8.241.195:37471)
0806 18:43:08.318282 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.318293 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.318315 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:43:08.318384 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.318389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.318580 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.318734 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:08.319159 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.320241 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.321408 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.323958 (+ 2550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.324074 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.324213 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:08.324345 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 18:43:08.324442 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":144,"threads_started":1}
I20260806 18:43:08.324836 1063 raft_consensus.cc:348] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:08.324950 1063 raft_consensus.cc:374] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:08.324987 1063 raft_consensus.cc:729] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181bfe4e15e145b39095c438caccc52b, State: Initialized, Role: FOLLOWER
I20260806 18:43:08.325141 1063 consensus_queue.cc:260] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:08.325414 1063 ts_tablet_manager.cc:1428] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:43:08.325834 1346 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1428eb193341719ab670859093b3be (DEFAULT_TABLE table=test-table [id=38f5fae8bf8647c4855921a0f7bdba14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:08.326990 1346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1428eb193341719ab670859093b3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:08.327028 1543 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1428eb193341719ab670859093b3be (DEFAULT_TABLE table=test-table [id=38f5fae8bf8647c4855921a0f7bdba14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:08.327977 1543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1428eb193341719ab670859093b3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:08.333876 1660 tablet_bootstrap.cc:492] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: Bootstrap starting.
I20260806 18:43:08.334086 1662 tablet_bootstrap.cc:492] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: Bootstrap starting.
I20260806 18:43:08.336112 1662 tablet_bootstrap.cc:654] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:08.336431 1660 tablet_bootstrap.cc:654] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:08.337148 1660 log.cc:824] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:08.337208 1662 log.cc:824] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:08.338855 1660 tablet_bootstrap.cc:492] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: No bootstrap required, opened a new log
I20260806 18:43:08.338876 1662 tablet_bootstrap.cc:492] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: No bootstrap required, opened a new log
I20260806 18:43:08.339032 1660 ts_tablet_manager.cc:1397] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 18:43:08.339046 1662 ts_tablet_manager.cc:1397] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:43:08.342104 1660 raft_consensus.cc:348] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:08.342135 1662 raft_consensus.cc:348] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:08.342331 1660 raft_consensus.cc:374] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:08.342346 1662 raft_consensus.cc:374] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:08.342368 1660 raft_consensus.cc:729] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27db750cc47c45beb5690eb887b5b57a, State: Initialized, Role: FOLLOWER
I20260806 18:43:08.342406 1662 raft_consensus.cc:729] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad1e7bcc4c1428193386df77cc6b74a, State: Initialized, Role: FOLLOWER
I20260806 18:43:08.342772 1660 consensus_queue.cc:260] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:08.342778 1662 consensus_queue.cc:260] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:08.343403 1640 heartbeater.cc:503] Master 127.8.241.254:39103 was elected leader, sending a full tablet report...
I20260806 18:43:08.343600 1446 heartbeater.cc:503] Master 127.8.241.254:39103 was elected leader, sending a full tablet report...
I20260806 18:43:08.343672 1660 ts_tablet_manager.cc:1428] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260806 18:43:08.343672 1662 ts_tablet_manager.cc:1428] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 18:43:08.346148 1447 tablet.cc:2369] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:43:08.378763 1231 tablet.cc:2369] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:08.402330 1668 raft_consensus.cc:482] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:08.402557 1668 raft_consensus.cc:504] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:08.403771 1668 leader_election.cc:288] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097), 181bfe4e15e145b39095c438caccc52b (127.8.241.193:40419)
I20260806 18:43:08.407677 1643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.403973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36097 (local address 127.8.241.195:44127)
0806 18:43:08.404188 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:08.404201 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:08.404295 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:08.404606 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:08.404608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:08.404624 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:08.404779 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.404787 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.405599 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.405602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.406531 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.406542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.407258 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.407262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.407371 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.407397 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:43:08.407468 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:43:08.407537 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":26}
I20260806 18:43:08.407703 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.404096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:44127 (local address 127.8.241.194:36097)
0806 18:43:08.404223 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.404228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.404243 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:43:08.404439 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.404442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.404503 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.404586 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:08.404920 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.405489 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.406662 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.407139 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.407234 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.407276 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:08.407498 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 18:43:08.407573 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:43:08.407791 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.404153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:44325 (local address 127.8.241.193:40419)
0806 18:43:08.404277 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.404281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.404489 (+ 208us) server_negotiation.cc:407] Connection header received
0806 18:43:08.404586 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.404589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.404653 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.404764 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:08.405070 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.405618 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.406733 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.407245 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.407369 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.407528 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:08.407612 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 18:43:08.407682 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:43:08.407835 1671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.404063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:40419 (local address 127.8.241.195:44325)
0806 18:43:08.404411 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:08.404421 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:08.404497 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:08.404781 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:08.404785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:08.404804 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:08.404934 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.404942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.405762 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.405766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.406604 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:08.406617 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.407370 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.407373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:08.407486 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.407513 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:43:08.407584 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:43:08.407651 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":29,"thread_start_us":125,"threads_started":1}
I20260806 18:43:08.408349 1161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1428eb193341719ab670859093b3be" candidate_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181bfe4e15e145b39095c438caccc52b" is_pre_election: true
I20260806 18:43:08.408417 1373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1428eb193341719ab670859093b3be" candidate_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27db750cc47c45beb5690eb887b5b57a" is_pre_election: true
I20260806 18:43:08.408638 1161 raft_consensus.cc:2454] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad1e7bcc4c1428193386df77cc6b74a in term 0.
I20260806 18:43:08.408710 1373 raft_consensus.cc:2454] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad1e7bcc4c1428193386df77cc6b74a in term 0.
I20260806 18:43:08.409133 1470 leader_election.cc:302] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 181bfe4e15e145b39095c438caccc52b, 6ad1e7bcc4c1428193386df77cc6b74a; no voters:
I20260806 18:43:08.409418 1668 raft_consensus.cc:2781] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:43:08.409498 1668 raft_consensus.cc:482] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:43:08.409528 1668 raft_consensus.cc:3037] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:08.410569 1668 raft_consensus.cc:504] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:08.410929 1668 leader_election.cc:288] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [CANDIDATE]: Term 1 election: Requested vote from peers 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097), 181bfe4e15e145b39095c438caccc52b (127.8.241.193:40419)
I20260806 18:43:08.411264 1373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1428eb193341719ab670859093b3be" candidate_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27db750cc47c45beb5690eb887b5b57a"
I20260806 18:43:08.411301 1161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1428eb193341719ab670859093b3be" candidate_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181bfe4e15e145b39095c438caccc52b"
I20260806 18:43:08.411368 1373 raft_consensus.cc:3037] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:08.411417 1161 raft_consensus.cc:3037] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:08.412379 1373 raft_consensus.cc:2454] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1e7bcc4c1428193386df77cc6b74a in term 1.
I20260806 18:43:08.412503 1161 raft_consensus.cc:2454] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad1e7bcc4c1428193386df77cc6b74a in term 1.
I20260806 18:43:08.412686 1471 leader_election.cc:302] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27db750cc47c45beb5690eb887b5b57a, 6ad1e7bcc4c1428193386df77cc6b74a; no voters:
I20260806 18:43:08.412912 1668 raft_consensus.cc:2781] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:08.413146 1668 raft_consensus.cc:686] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 1 LEADER]: Becoming Leader. State: Replica: 6ad1e7bcc4c1428193386df77cc6b74a, State: Running, Role: LEADER
I20260806 18:43:08.413456 1668 consensus_queue.cc:237] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:08.416710 950 catalog_manager.cc:5495] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195). New cstate: current_term: 1 leader_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } health_report { overall_health: UNKNOWN } } }
I20260806 18:43:08.446257 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.442816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60844 (local address 127.8.241.195:37471)
0806 18:43:08.442943 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:08.442946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:08.442960 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:43:08.443062 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:08.443064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:08.443119 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:08.443203 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:43:08.443389 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.443916 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.444698 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:08.444980 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:08.445099 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:08.445401 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:43:08.445867 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:43:08.445973 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:08.446052 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 18:43:08.446111 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:43:08.452428 1161 raft_consensus.cc:1261] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Refusing update from remote peer 6ad1e7bcc4c1428193386df77cc6b74a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:08.452447 1373 raft_consensus.cc:1261] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [term 1 FOLLOWER]: Refusing update from remote peer 6ad1e7bcc4c1428193386df77cc6b74a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:08.453174 1668 consensus_queue.cc:1035] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:08.453413 1672 consensus_queue.cc:1035] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [LEADER]: Connected to new peer: Peer: permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:08.460088 1675 mvcc.cc:205] Tried to move back new op lower bound from 7315627165493178368 to 7315627165344100352. Current Snapshot: MvccSnapshot[applied={T|T < 7315627165493178368}]
I20260806 18:43:08.460520 1677 mvcc.cc:205] Tried to move back new op lower bound from 7315627165493178368 to 7315627165344100352. Current Snapshot: MvccSnapshot[applied={T|T < 7315627165493178368}]
I20260806 18:43:08.469357 948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:08.469512 948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:08.471808 948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Sending DeleteTablet for 3 replicas of tablet 4a1428eb193341719ab670859093b3be
I20260806 18:43:08.472460 1346 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1428eb193341719ab670859093b3be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:08 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:43800
I20260806 18:43:08.472602 1543 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1428eb193341719ab670859093b3be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:08 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:60828
I20260806 18:43:08.472932 1691 tablet_replica.cc:330] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: stopping tablet replica
I20260806 18:43:08.473241 1123 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1428eb193341719ab670859093b3be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:08 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:44510
I20260806 18:43:08.473336 1691 raft_consensus.cc:2229] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:08.473366 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 1001
I20260806 18:43:08.473124 1692 tablet_replica.cc:330] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: stopping tablet replica
I20260806 18:43:08.483160 1692 raft_consensus.cc:2229] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:43:08.483711 1692 raft_consensus.cc:2258] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:08.490613 1692 ts_tablet_manager.cc:1905] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:08.498909 1692 ts_tablet_manager.cc:1918] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:08.499143 1692 log.cc:1196] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/4a1428eb193341719ab670859093b3be
I20260806 18:43:08.499734 1692 ts_tablet_manager.cc:1939] T 4a1428eb193341719ab670859093b3be P 6ad1e7bcc4c1428193386df77cc6b74a: Deleting consensus metadata
I20260806 18:43:08.502048 927 catalog_manager.cc:4841] TS 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471): tablet 4a1428eb193341719ab670859093b3be (table test-table [id=38f5fae8bf8647c4855921a0f7bdba14]) successfully deleted
I20260806 18:43:08.506659 1691 raft_consensus.cc:2258] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:08.508515 1691 ts_tablet_manager.cc:1905] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:08.511497 1691 ts_tablet_manager.cc:1918] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:08.511701 1691 log.cc:1196] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/4a1428eb193341719ab670859093b3be
I20260806 18:43:08.512162 1691 ts_tablet_manager.cc:1939] T 4a1428eb193341719ab670859093b3be P 27db750cc47c45beb5690eb887b5b57a: Deleting consensus metadata
I20260806 18:43:08.513705 931 catalog_manager.cc:4841] TS 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097): tablet 4a1428eb193341719ab670859093b3be (table test-table [id=38f5fae8bf8647c4855921a0f7bdba14]) successfully deleted
W20260806 18:43:08.515445 929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:40419 (error 108)
I20260806 18:43:08.516101 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 1239
I20260806 18:43:08.520673 1652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:08.520301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:40419 (local address 127.0.0.1:44522)
0806 18:43:08.520506 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:08.520580 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:40419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260806 18:43:08.520856 929 catalog_manager.cc:4584] TS 181bfe4e15e145b39095c438caccc52b (127.8.241.193:40419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a1428eb193341719ab670859093b3be: Network error: Client connection negotiation failed: client connection to 127.8.241.193:40419: connect: Connection refused (error 111)
I20260806 18:43:08.528236 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 1449
I20260806 18:43:08.550772 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 906
I20260806 18:43:28.563724 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39103
--webserver_interface=127.8.241.254
--webserver_port=33977
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:28.682641 1696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:28.682955 1696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:28.683000 1696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:28.687270 1696 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:43:28.687343 1696 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:43:28.687361 1696 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:43:28.687377 1696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:28.687397 1696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:28.687422 1696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:28.687458 1696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:28.687489 1696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:28.693071 1696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=33977
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:28.694239 1696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:28.695339 1696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:28.704336 1701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:28.704341 1702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:28.704295 1705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:28.704643 1696 server_base.cc:1034] running on GCE node
I20260806 18:43:28.705302 1696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:28.706472 1696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:28.707676 1696 hybrid_clock.cc:648] HybridClock initialized: now 1786041808707655 us; error 31 us; skew 500 ppm
Aug 06 18:43:28 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041808, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:28.713994 1696 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:43:28.714974 1696 webserver.cc:460] Webserver started at http://127.8.241.254:33977/ using document root <none> and password file <none>
I20260806 18:43:28.715528 1696 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:28.715587 1696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:28.719638 1696 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:43:28.721936 1714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:28.723240 1696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:43:28.723366 1696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "a9aafb8d917a40218b9f38a792af6d31"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "62b20c342a74bde5d35c7b5d9151e64e"
server_key_iv: "45a939d8ef29cb01dac82f8d4cf5fb92"
server_key_version: "encryptionkey@0"
I20260806 18:43:28.723951 1696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:28.741312 1696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:28.744167 1696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:28.744385 1696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:28.751863 1696 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:39103
I20260806 18:43:28.752146 1795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:39103 every 8 connection(s)
I20260806 18:43:28.753039 1696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:28.757566 1796 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:43:28.759258 1796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Bootstrap starting.
I20260806 18:43:28.761246 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 1696
I20260806 18:43:28.761735 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:40419
--local_ip_for_outbound_sockets=127.8.241.193
--tserver_master_addrs=127.8.241.254:39103
--webserver_port=40949
--webserver_interface=127.8.241.193
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:43:28.778000 1796 log.cc:824] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:28.791025 1796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:43:28.791416 1796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: Bootstrap complete.
I20260806 18:43:28.795859 1796 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:43:28.796262 1796 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9aafb8d917a40218b9f38a792af6d31, State: Initialized, Role: FOLLOWER
I20260806 18:43:28.796805 1796 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:43:28.796937 1796 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:28.797019 1796 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:28.797125 1796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:43:28.799003 1796 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:43:28.799345 1796 leader_election.cc:302] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9aafb8d917a40218b9f38a792af6d31; no voters:
I20260806 18:43:28.799652 1796 leader_election.cc:288] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:43:28.799821 1802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:43:28.800077 1802 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [term 3 LEADER]: Becoming Leader. State: Replica: a9aafb8d917a40218b9f38a792af6d31, State: Running, Role: LEADER
I20260806 18:43:28.800428 1802 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } }
I20260806 18:43:28.800984 1796 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:28.801684 1802 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9aafb8d917a40218b9f38a792af6d31. Latest consensus state: current_term: 3 leader_uuid: "a9aafb8d917a40218b9f38a792af6d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } } }
I20260806 18:43:28.802459 1802 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:28.802347 1803 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a9aafb8d917a40218b9f38a792af6d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9aafb8d917a40218b9f38a792af6d31" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39103 } } }
I20260806 18:43:28.803176 1803 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:28.806638 1824 catalog_manager.cc:1259] Loaded cluster ID: 4a43eb72728d421aaa81913a02fedd50
I20260806 18:43:28.806717 1824 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: loading cluster ID for follower catalog manager: success
I20260806 18:43:28.809353 1824 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a9aafb8d917a40218b9f38a792af6d31: acquiring CA information for follower catalog manager: success
F20260806 18:43:28.810678 1824 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fcf5870e9ad google::LogMessage::Fail() at ??:0
@ 0x7fcf5871188a google::LogMessage::SendToLog() at ??:0
@ 0x7fcf5870e4de google::LogMessage::Flush() at ??:0
@ 0x7fcf5870fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fcf5b3d218d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fcf63073989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7fcf63050023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7fcf6305054b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7fcf63049652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7fcf630489e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7fcf6308f541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fcf5a951512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fcf5bf956db start_thread at ??:0
@ 0x7fcf5720c61f clone at ??:0
W20260806 18:43:28.924547 1800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:28.924868 1800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:28.924944 1800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:28.929258 1800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:28.929338 1800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:28.929438 1800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:43:28.934998 1800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:40419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=40949
--tserver_master_addrs=127.8.241.254:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:28.936192 1800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:28.937424 1800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:28.948625 1851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:28.948798 1800 server_base.cc:1034] running on GCE node
W20260806 18:43:28.948619 1852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:28.948618 1855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:28.949568 1800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:28.950372 1800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:28.951570 1800 hybrid_clock.cc:648] HybridClock initialized: now 1786041808951535 us; error 44 us; skew 500 ppm
Aug 06 18:43:28 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041808, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:28.955170 1800 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:43:28.956415 1800 webserver.cc:460] Webserver started at http://127.8.241.193:40949/ using document root <none> and password file <none>
I20260806 18:43:28.957186 1800 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:28.957298 1800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:28.962622 1800 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:43:28.965545 1862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:28.967141 1800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:43:28.967281 1800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "181bfe4e15e145b39095c438caccc52b"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "3b692be522385de50bdaacca05b52d9b"
server_key_iv: "905e38dfedd112d9318b7fae9c6cfc13"
server_key_version: "encryptionkey@0"
I20260806 18:43:28.968075 1800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:28.986711 1800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:28.990896 1800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:28.991114 1800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:28.991932 1800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:28.993422 1871 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:43:28.997887 1800 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:43:28.997967 1800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 18:43:28.998039 1800 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:43:29.000425 1800 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:43:29.000492 1800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:43:29.000873 1871 tablet_bootstrap.cc:492] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Bootstrap starting.
I20260806 18:43:29.011870 1800 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:40419
I20260806 18:43:29.012147 2027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:40419 every 8 connection(s)
I20260806 18:43:29.013216 1800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:43:29.017904 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 1800
I20260806 18:43:29.018422 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:36097
--local_ip_for_outbound_sockets=127.8.241.194
--tserver_master_addrs=127.8.241.254:39103
--webserver_port=35117
--webserver_interface=127.8.241.194
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:43:29.022010 1871 log.cc:824] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:29.031944 1871 tablet_bootstrap.cc:492] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:43:29.032410 1871 tablet_bootstrap.cc:492] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Bootstrap complete.
I20260806 18:43:29.033974 1871 ts_tablet_manager.cc:1397] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.011s
I20260806 18:43:29.037384 1871 raft_consensus.cc:348] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:29.037953 1871 raft_consensus.cc:729] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181bfe4e15e145b39095c438caccc52b, State: Initialized, Role: FOLLOWER
I20260806 18:43:29.038522 1871 consensus_queue.cc:260] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:29.039646 1871 ts_tablet_manager.cc:1428] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260806 18:43:29.068166 2031 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:43:29.015666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:34033)
0806 18:43:29.016247 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:29.016340 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:29.017576 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:29.067165 (+ 49589us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":136,"threads_started":1}
W20260806 18:43:29.068760 2028 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:39103 (0 consecutive failures): Network error: Failed to ping master at 127.8.241.254:39103: Client connection negotiation failed: client connection to 127.8.241.254:39103: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:43:29.069562 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:29.069284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:41087)
0806 18:43:29.069421 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:29.069496 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:43:29.070158 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:29.069957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:44745)
0806 18:43:29.070056 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:29.070105 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:43:29.070325 2028 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:43:29.144866 2032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:29.145160 2032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:29.145208 2032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:29.149176 2032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:29.149236 2032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:29.149293 2032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:43:29.154698 2032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:36097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=35117
--tserver_master_addrs=127.8.241.254:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:29.155769 2032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:29.156814 2032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:29.166579 2050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:29.166486 2044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:29.166693 2032 server_base.cc:1034] running on GCE node
W20260806 18:43:29.166483 2045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:29.167296 2032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:29.168001 2032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:29.169163 2032 hybrid_clock.cc:648] HybridClock initialized: now 1786041809169130 us; error 43 us; skew 500 ppm
Aug 06 18:43:29 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041809, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:29.171846 2032 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:43:29.172713 2032 webserver.cc:460] Webserver started at http://127.8.241.194:35117/ using document root <none> and password file <none>
I20260806 18:43:29.173328 2032 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:29.173416 2032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:29.177171 2032 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:43:29.179122 2057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:29.180266 2032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:29.180394 2032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "27db750cc47c45beb5690eb887b5b57a"
format_stamp: "Formatted at 2026-08-06 18:42:46 on dist-test-slave-k5rr"
server_key: "73f8d8e7c4cb4dccabaf90393b403f79"
server_key_iv: "bf178f7be636608156e93e57a987704a"
server_key_version: "encryptionkey@0"
I20260806 18:43:29.180917 2032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:29.192695 2032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:29.195459 2032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:29.195631 2032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:29.196200 2032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:29.197055 2032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:29.197101 2032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:29.197154 2032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:29.197182 2032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:29.205852 2032 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:36097
I20260806 18:43:29.205881 2223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:36097 every 8 connection(s)
I20260806 18:43:29.206857 2032 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:43:29.207262 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 2032
I20260806 18:43:29.207758 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:37471
--local_ip_for_outbound_sockets=127.8.241.195
--tserver_master_addrs=127.8.241.254:39103
--webserver_port=40633
--webserver_interface=127.8.241.195
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:43:29.211360 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:29.209172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:47761)
0806 18:43:29.209704 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:29.209861 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":128,"threads_started":1}
W20260806 18:43:29.211814 2224 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:39103 (0 consecutive failures): Network error: Failed to ping master at 127.8.241.254:39103: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
I20260806 18:43:29.212445 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:29.212203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:53707)
0806 18:43:29.212325 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:29.212381 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 18:43:29.213124 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:29.212847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:53721)
0806 18:43:29.212984 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:29.213062 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260806 18:43:29.213297 2224 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:43:29.326515 2226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:29.326769 2226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:29.326810 2226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:29.330739 2226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:29.330796 2226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:29.330852 2226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:43:29.335927 2226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:37471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=40633
--tserver_master_addrs=127.8.241.254:39103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:29.336915 2226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:29.337918 2226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:29.347257 2239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:29.347266 2236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:29.347266 2235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:29.347537 2226 server_base.cc:1034] running on GCE node
I20260806 18:43:29.347936 2226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:29.348506 2226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:29.349666 2226 hybrid_clock.cc:648] HybridClock initialized: now 1786041809349646 us; error 29 us; skew 500 ppm
Aug 06 18:43:29 dist-test-slave-k5rr krb5kdc[32555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041809, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:29.352365 2226 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:43:29.353242 2226 webserver.cc:460] Webserver started at http://127.8.241.195:40633/ using document root <none> and password file <none>
I20260806 18:43:29.353809 2226 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:29.353878 2226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:29.357709 2226 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:43:29.359774 2252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:29.361143 2226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:43:29.361258 2226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "6ad1e7bcc4c1428193386df77cc6b74a"
format_stamp: "Formatted at 2026-08-06 18:42:47 on dist-test-slave-k5rr"
server_key: "eaaa0be5a7a7004914fa30c7a9ade340"
server_key_iv: "2be56a2587f383ab330e194ee191ea8b"
server_key_version: "encryptionkey@0"
I20260806 18:43:29.361829 2226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:29.410341 2226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:29.413516 2226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:29.413714 2226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:29.414351 2226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:29.415223 2226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:29.415271 2226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:29.415324 2226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:29.415349 2226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:29.424491 2226 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:37471
I20260806 18:43:29.424549 2417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:37471 every 8 connection(s)
I20260806 18:43:29.425587 2226 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:43:29.426806 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 2226
I20260806 18:43:29.429847 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:29.428047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:42663)
0806 18:43:29.428530 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:29.428634 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":129,"threads_started":1}
W20260806 18:43:29.430289 2418 heartbeater.cc:644] Failed to heartbeat to 127.8.241.254:39103 (0 consecutive failures): Network error: Failed to ping master at 127.8.241.254:39103: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
I20260806 18:43:29.430872 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:29.430658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:36321)
0806 18:43:29.430763 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:29.430812 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 18:43:29.431423 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:29.431221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:46935)
0806 18:43:29.431314 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:29.431376 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 18:43:29.431593 2418 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:43:30.071398 2424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:30.070932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:40559)
0806 18:43:30.071226 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:30.071288 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":94,"threads_started":1}
I20260806 18:43:30.214316 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:30.213835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:59579)
0806 18:43:30.214129 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:30.214187 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":73,"threads_started":1}
I20260806 18:43:30.432554 2429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:30.432146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:47707)
0806 18:43:30.432383 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:30.432438 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":72,"threads_started":1}
I20260806 18:43:30.679777 2430 raft_consensus.cc:482] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:30.679992 2430 raft_consensus.cc:504] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:30.681224 2430 leader_election.cc:288] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097), 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471)
Aug 06 18:43:30 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041808, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.194@KRBTEST.COM
Aug 06 18:43:30 dist-test-slave-k5rr krb5kdc[32555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041808, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.195@KRBTEST.COM
I20260806 18:43:30.699049 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:30.681429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:36097 (local address 127.8.241.193:44611)
0806 18:43:30.681857 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:30.681869 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:30.681930 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:30.684002 (+ 2072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:30.684009 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:30.684658 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:30.685567 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:30.685592 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:30.687539 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:30.687542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:30.689233 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:30.689245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:30.690308 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:30.690312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:30.690598 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:30.691448 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:30.691476 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:30.693390 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:30.695771 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:30.695778 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:30.695795 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:30.696266 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:30.696690 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:30.696693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:30.696695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:30.696950 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:30.697844 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:30.697851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:30.698235 (+ 384us) client_negotiation.cc:769] Sending connection context
0806 18:43:30.698447 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 18:43:30.698823 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":352,"spinlock_wait_cycles":36480,"thread_start_us":72,"threads_started":1}
I20260806 18:43:30.699049 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:30.681429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37471 (local address 127.8.241.193:45151)
0806 18:43:30.681757 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:30.681774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:30.681900 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:30.684002 (+ 2102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:30.684009 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:30.684658 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:30.685567 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:30.685592 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:30.687534 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:30.687538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:30.689232 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:30.689245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:30.690308 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:30.690312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:30.690598 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:30.691437 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:30.691480 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:30.693668 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:30.695771 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:30.695778 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:30.695795 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:30.696266 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:30.696690 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:30.696693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:30.696695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:30.696950 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:30.697844 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:30.697851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:30.698235 (+ 384us) client_negotiation.cc:769] Sending connection context
0806 18:43:30.698447 (+ 212us) client_negotiation.cc:240] Negotiation successful
0806 18:43:30.698885 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":594,"spinlock_wait_cycles":28672,"thread_start_us":67,"threads_started":1}
I20260806 18:43:30.699187 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:30.681629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:45151 (local address 127.8.241.195:37471)
0806 18:43:30.682055 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:30.682082 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:30.682128 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:43:30.683059 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:30.683067 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:30.683532 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:30.683892 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:30.685801 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:30.687357 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:30.689389 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:30.690161 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:30.690372 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:30.693868 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:30.693882 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:30.693899 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:30.693934 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:30.695646 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:30.696402 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:30.696409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:30.696415 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:30.696592 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:30.697110 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:30.697114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:30.697115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:30.697727 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:30.697875 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:30.698644 (+ 769us) server_negotiation.cc:299] Negotiation successful
0806 18:43:30.698975 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":99,"threads_started":1}
I20260806 18:43:30.699734 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:30.681629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:44611 (local address 127.8.241.194:36097)
0806 18:43:30.682055 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:30.682082 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:30.682128 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:43:30.683059 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:30.683067 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:30.683532 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:30.683892 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:30.685765 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:30.687357 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:30.689389 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:30.690161 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:30.690372 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:30.693633 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:30.693639 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:30.693653 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:30.693682 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:30.695646 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:30.696402 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:30.696409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:30.696414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:30.696592 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:30.697120 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:30.697124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:30.697125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:30.697719 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:30.697875 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:30.698644 (+ 769us) server_negotiation.cc:299] Negotiation successful
0806 18:43:30.698975 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":92,"threads_started":1}
W20260806 18:43:30.701930 1866 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
W20260806 18:43:30.702114 1863 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
I20260806 18:43:30.702183 1863 leader_election.cc:302] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 181bfe4e15e145b39095c438caccc52b; no voters: 27db750cc47c45beb5690eb887b5b57a, 6ad1e7bcc4c1428193386df77cc6b74a
I20260806 18:43:30.702471 2430 raft_consensus.cc:2726] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:43:31.072530 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:31.072260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:47279)
0806 18:43:31.072390 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:31.072442 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:43:31.215372 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:31.214985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:44071)
0806 18:43:31.215211 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:31.215261 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":67,"threads_started":1}
I20260806 18:43:31.433665 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:31.433247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:33309)
0806 18:43:31.433498 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:31.433555 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260806 18:43:32.073699 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:32.073250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:42455)
0806 18:43:32.073533 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:32.073595 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
I20260806 18:43:32.216583 2446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:32.216138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:36425)
0806 18:43:32.216414 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:32.216477 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
I20260806 18:43:32.434902 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:32.434416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:38639)
0806 18:43:32.434710 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:32.434800 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":96,"threads_started":1}
I20260806 18:43:32.723026 2449 raft_consensus.cc:482] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:32.723167 2449 raft_consensus.cc:504] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:32.723645 2449 leader_election.cc:288] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097), 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471)
W20260806 18:43:32.724442 1866 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
W20260806 18:43:32.724536 1863 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
I20260806 18:43:32.724653 1863 leader_election.cc:302] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 181bfe4e15e145b39095c438caccc52b; no voters: 27db750cc47c45beb5690eb887b5b57a, 6ad1e7bcc4c1428193386df77cc6b74a
I20260806 18:43:32.724850 2449 raft_consensus.cc:2726] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:43:33.074899 2450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:33.074430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:39481)
0806 18:43:33.074706 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:33.074770 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":79,"threads_started":1}
I20260806 18:43:33.217821 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:33.217391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:43991)
0806 18:43:33.217660 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:33.217726 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
I20260806 18:43:33.436302 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:33.435766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:34945)
0806 18:43:33.436099 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:33.436182 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":89,"threads_started":1}
I20260806 18:43:34.076205 2453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:34.075764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:46263)
0806 18:43:34.076050 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:34.076106 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":75,"threads_started":1}
I20260806 18:43:34.218977 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:34.218545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:47521)
0806 18:43:34.218823 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:34.218880 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":75,"threads_started":1}
I20260806 18:43:34.437511 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:34.437111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:37831)
0806 18:43:34.437346 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:34.437413 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260806 18:43:35.077417 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:35.076943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:42533)
0806 18:43:35.077250 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:35.077312 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":106,"threads_started":1}
I20260806 18:43:35.220172 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:35.219741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:60411)
0806 18:43:35.220009 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:35.220066 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":79,"threads_started":1}
I20260806 18:43:35.438831 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:35.438327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:59681)
0806 18:43:35.438658 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:35.438718 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":93,"threads_started":1}
I20260806 18:43:35.441123 2459 raft_consensus.cc:482] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:35.441253 2459 raft_consensus.cc:504] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:35.441644 2459 leader_election.cc:288] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097), 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471)
W20260806 18:43:35.442412 1863 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
W20260806 18:43:35.442525 1866 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
I20260806 18:43:35.442680 1866 leader_election.cc:302] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 181bfe4e15e145b39095c438caccc52b; no voters: 27db750cc47c45beb5690eb887b5b57a, 6ad1e7bcc4c1428193386df77cc6b74a
I20260806 18:43:35.442828 2459 raft_consensus.cc:2726] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:43:36.078603 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:36.078134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:46545)
0806 18:43:36.078430 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:36.078494 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":74,"threads_started":1}
I20260806 18:43:36.221390 2461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:36.220916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:40005)
0806 18:43:36.221232 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:36.221299 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":111,"threads_started":1}
I20260806 18:43:36.440016 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:36.439569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:50115)
0806 18:43:36.439865 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:36.439922 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":77,"threads_started":1}
I20260806 18:43:37.079798 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:37.079392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:58945)
0806 18:43:37.079640 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:37.079699 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260806 18:43:37.222537 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:37.222117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:36171)
0806 18:43:37.222375 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:37.222434 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":75,"threads_started":1}
I20260806 18:43:37.441145 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:37.440672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:56877)
0806 18:43:37.440951 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:37.441040 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
I20260806 18:43:38.081058 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:38.080593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:57771)
0806 18:43:38.080856 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:38.080919 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
I20260806 18:43:38.223819 2468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:38.223318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:56887)
0806 18:43:38.223622 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:38.223683 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":113,"threads_started":1}
I20260806 18:43:38.442265 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:38.441830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:58777)
0806 18:43:38.442089 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:38.442146 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":89,"threads_started":1}
I20260806 18:43:38.727640 2470 raft_consensus.cc:482] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:38.727819 2470 raft_consensus.cc:504] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:38.728248 2470 leader_election.cc:288] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097), 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471)
W20260806 18:43:38.729048 1863 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
W20260806 18:43:38.729153 1866 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
I20260806 18:43:38.729249 1866 leader_election.cc:302] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 181bfe4e15e145b39095c438caccc52b; no voters: 27db750cc47c45beb5690eb887b5b57a, 6ad1e7bcc4c1428193386df77cc6b74a
I20260806 18:43:38.729398 2470 raft_consensus.cc:2726] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:43:39.082306 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:39.081912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:38833)
0806 18:43:39.082153 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:39.082208 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":72,"threads_started":1}
I20260806 18:43:39.225204 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:39.224686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:57861)
0806 18:43:39.225032 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:39.225099 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
I20260806 18:43:39.443420 2473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:39.442996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:56883)
0806 18:43:39.443249 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:39.443305 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":83,"threads_started":1}
I20260806 18:43:40.083470 2474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:40.083042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:46917)
0806 18:43:40.083295 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:40.083366 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
I20260806 18:43:40.226497 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:40.226006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:35093)
0806 18:43:40.226306 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:40.226372 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":75,"threads_started":1}
I20260806 18:43:40.444531 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:40.444105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:56781)
0806 18:43:40.444375 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:40.444433 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260806 18:43:41.084714 2480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:41.084252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:51267)
0806 18:43:41.084531 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:41.084593 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
I20260806 18:43:41.227720 2481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:41.227296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:57045)
0806 18:43:41.227564 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:41.227621 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":94,"threads_started":1}
I20260806 18:43:41.445670 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:41.445252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:59117)
0806 18:43:41.445512 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:41.445569 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":95,"threads_started":1}
I20260806 18:43:42.085843 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:42.085435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:46355)
0806 18:43:42.085689 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:42.085743 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
I20260806 18:43:42.228916 2484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:42.228489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:33575)
0806 18:43:42.228730 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:42.228815 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260806 18:43:42.446923 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:42.446482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:58095)
0806 18:43:42.446742 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:42.446803 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":84,"threads_started":1}
I20260806 18:43:43.086956 2486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:43.086531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:53597)
0806 18:43:43.086785 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:43.086836 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
I20260806 18:43:43.230120 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:43.229669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:45919)
0806 18:43:43.229943 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:43.230011 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260806 18:43:43.448022 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:43.447620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:60941)
0806 18:43:43.447868 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:43.447924 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
I20260806 18:43:43.638886 2490 raft_consensus.cc:482] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:43.639030 2490 raft_consensus.cc:504] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db750cc47c45beb5690eb887b5b57a" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 36097 } } peers { permanent_uuid: "6ad1e7bcc4c1428193386df77cc6b74a" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37471 } } peers { permanent_uuid: "181bfe4e15e145b39095c438caccc52b" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 40419 } }
I20260806 18:43:43.639456 2490 leader_election.cc:288] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097), 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471)
W20260806 18:43:43.640318 1863 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ad1e7bcc4c1428193386df77cc6b74a (127.8.241.195:37471): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
W20260806 18:43:43.640448 1866 leader_election.cc:341] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db750cc47c45beb5690eb887b5b57a (127.8.241.194:36097): Not found: Tablet not found: 4a1428eb193341719ab670859093b3be
I20260806 18:43:43.640520 1866 leader_election.cc:302] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 181bfe4e15e145b39095c438caccc52b; no voters: 27db750cc47c45beb5690eb887b5b57a, 6ad1e7bcc4c1428193386df77cc6b74a
I20260806 18:43:43.640668 2490 raft_consensus.cc:2726] T 4a1428eb193341719ab670859093b3be P 181bfe4e15e145b39095c438caccc52b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:43:44.088287 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:44.087884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.193:46765)
0806 18:43:44.088126 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:44.088181 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
I20260806 18:43:44.231488 2492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:44.231015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.194:60189)
0806 18:43:44.231299 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:44.231365 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
I20260806 18:43:44.436617 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 1800
I20260806 18:43:44.448935 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 2032
I20260806 18:43:44.452714 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:44.448950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39103 (local address 127.8.241.195:60325)
0806 18:43:44.452468 (+ 3518us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:44.452555 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.254:39103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3409,"thread_start_us":102,"threads_started":1}
I20260806 18:43:44.468348 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 2226
2026-08-06T18:43:44Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59447 ms)
[----------] 30 tests from SecurityITest (246711 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:43:44 dist-test-slave-k5rr krb5kdc[2498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:43:44 dist-test-slave-k5rr krb5kdc[2498](info): set up 2 sockets
Aug 06 18:43:44 dist-test-slave-k5rr krb5kdc[2498](info): commencing operation
krb5kdc: starting...
W20260806 18:43:45.607887 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:43:45 dist-test-slave-k5rr krb5kdc[2498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:43:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:43:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:43:45.731431 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:41367
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:41367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:45.841617 2514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:45.841878 2514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:45.841935 2514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:45.845886 2514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:45.845938 2514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:45.845965 2514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:45.845986 2514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:45.846014 2514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:45.851104 2514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:41367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:41367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:45.852128 2514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:45.853117 2514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:45.861490 2529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:45.861526 2531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:45.861526 2528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:45.861582 2514 server_base.cc:1034] running on GCE node
I20260806 18:43:45.862264 2514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:45.863324 2514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:45.864486 2514 hybrid_clock.cc:648] HybridClock initialized: now 1786041825864467 us; error 31 us; skew 500 ppm
Aug 06 18:43:45 dist-test-slave-k5rr krb5kdc[2498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041825, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:45.867187 2514 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:43:45.868067 2514 webserver.cc:460] Webserver started at http://127.8.241.254:39513/ using document root <none> and password file <none>
I20260806 18:43:45.868602 2514 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:45.868660 2514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:45.868875 2514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:45.870898 2514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "2eee1fcf60e94f51848d46d64b9e95f6"
format_stamp: "Formatted at 2026-08-06 18:43:45 on dist-test-slave-k5rr"
server_key: "958a4c8f770a2a4383018fc46a936ab5"
server_key_iv: "963685cc4233b321000c282a7353a7ed"
server_key_version: "encryptionkey@0"
I20260806 18:43:45.871425 2514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "2eee1fcf60e94f51848d46d64b9e95f6"
format_stamp: "Formatted at 2026-08-06 18:43:45 on dist-test-slave-k5rr"
server_key: "958a4c8f770a2a4383018fc46a936ab5"
server_key_iv: "963685cc4233b321000c282a7353a7ed"
server_key_version: "encryptionkey@0"
I20260806 18:43:45.875025 2514 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:43:45.877414 2544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:45.878726 2514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:45.878850 2514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "2eee1fcf60e94f51848d46d64b9e95f6"
format_stamp: "Formatted at 2026-08-06 18:43:45 on dist-test-slave-k5rr"
server_key: "958a4c8f770a2a4383018fc46a936ab5"
server_key_iv: "963685cc4233b321000c282a7353a7ed"
server_key_version: "encryptionkey@0"
I20260806 18:43:45.879047 2514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:45.894255 2514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:45.897074 2514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:45.897262 2514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:45.904685 2514 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:41367
I20260806 18:43:45.904695 2628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:41367 every 8 connection(s)
I20260806 18:43:45.905709 2514 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:45.907361 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 2514
I20260806 18:43:45.907491 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:43:45.907877 9159 external_mini_cluster.cc:1428] Setting key bfa066a55d200069a92ba5ee40b9409f
I20260806 18:43:45.908910 2631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:45.914721 2631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6: Bootstrap starting.
Aug 06 18:43:45 dist-test-slave-k5rr krb5kdc[2498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:45.917116 2631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:45.917980 2631 log.cc:824] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:45.919920 2631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6: No bootstrap required, opened a new log
I20260806 18:43:45.921689 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:45.909375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.8.241.254:41367)
0806 18:43:45.909839 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:45.909856 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:45.909892 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:43:45.910551 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:45.910568 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:45.910889 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:45.911219 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:45.912091 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:45.913146 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:45.914024 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:45.914505 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:45.914674 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:45.917097 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:45.917103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:45.917116 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:45.917143 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:45.918846 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:45.919242 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:45.919248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:45.919255 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:45.919377 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:45.919628 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:45.919631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:45.919633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:45.920018 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:45.920179 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:45.920470 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 18:43:45.920737 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":149,"threads_started":1}
I20260806 18:43:45.923389 2631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eee1fcf60e94f51848d46d64b9e95f6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 41367 } }
I20260806 18:43:45.923607 2631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:45.923678 2631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eee1fcf60e94f51848d46d64b9e95f6, State: Initialized, Role: FOLLOWER
I20260806 18:43:45.924134 2631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eee1fcf60e94f51848d46d64b9e95f6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 41367 } }
I20260806 18:43:45.924302 2631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:45.924367 2631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:45.924463 2631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:45.925346 2631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eee1fcf60e94f51848d46d64b9e95f6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 41367 } }
I20260806 18:43:45.925666 2631 leader_election.cc:302] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eee1fcf60e94f51848d46d64b9e95f6; no voters:
I20260806 18:43:45.925951 2631 leader_election.cc:288] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:43:45.926036 2639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:45.926254 2639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [term 1 LEADER]: Becoming Leader. State: Replica: 2eee1fcf60e94f51848d46d64b9e95f6, State: Running, Role: LEADER
I20260806 18:43:45.926577 2639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eee1fcf60e94f51848d46d64b9e95f6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 41367 } }
I20260806 18:43:45.927132 2631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:45.928119 2641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eee1fcf60e94f51848d46d64b9e95f6. Latest consensus state: current_term: 1 leader_uuid: "2eee1fcf60e94f51848d46d64b9e95f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eee1fcf60e94f51848d46d64b9e95f6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 41367 } } }
I20260806 18:43:45.928256 2641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:45.928633 2640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eee1fcf60e94f51848d46d64b9e95f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eee1fcf60e94f51848d46d64b9e95f6" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 41367 } } }
I20260806 18:43:45.928840 2640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:45.928634 2649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:45.932121 2649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:45.937254 2649 catalog_manager.cc:1347] Generated new cluster ID: 4e4b81d6e52a4db99caf432ed8a33aab
I20260806 18:43:45.937318 2649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:45.951989 2649 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:43:45.953286 2649 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:45.958200 2649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6: Generated new TSK 0
I20260806 18:43:45.959034 2649 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:43:45.989631 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:41367
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:46.101617 2669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:46.101871 2669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:46.101930 2669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:46.105819 2669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:46.105880 2669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:46.105957 2669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:43:46.111002 2669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:41367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:46.112030 2669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:46.113030 2669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:46.122398 2680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:46.122411 2675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:46.122483 2677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:46.122575 2669 server_base.cc:1034] running on GCE node
I20260806 18:43:46.122949 2669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:46.123549 2669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:46.124704 2669 hybrid_clock.cc:648] HybridClock initialized: now 1786041826124661 us; error 57 us; skew 500 ppm
Aug 06 18:43:46 dist-test-slave-k5rr krb5kdc[2498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041826, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:46.127377 2669 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:43:46.128249 2669 webserver.cc:460] Webserver started at http://127.8.241.193:37699/ using document root <none> and password file <none>
I20260806 18:43:46.128819 2669 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:46.128899 2669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:46.129148 2669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:46.131158 2669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "855ba3de6fde42cebc2eae5b3e23392d"
format_stamp: "Formatted at 2026-08-06 18:43:46 on dist-test-slave-k5rr"
server_key: "ab77bc60c6ea9ab34d8b3fb51aa91862"
server_key_iv: "c4a1e9f8a87c0c2138850f33f4c2d846"
server_key_version: "encryptionkey@0"
I20260806 18:43:46.131719 2669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "855ba3de6fde42cebc2eae5b3e23392d"
format_stamp: "Formatted at 2026-08-06 18:43:46 on dist-test-slave-k5rr"
server_key: "ab77bc60c6ea9ab34d8b3fb51aa91862"
server_key_iv: "c4a1e9f8a87c0c2138850f33f4c2d846"
server_key_version: "encryptionkey@0"
I20260806 18:43:46.135327 2669 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 18:43:46.137661 2693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:46.138888 2669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:43:46.138996 2669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "855ba3de6fde42cebc2eae5b3e23392d"
format_stamp: "Formatted at 2026-08-06 18:43:46 on dist-test-slave-k5rr"
server_key: "ab77bc60c6ea9ab34d8b3fb51aa91862"
server_key_iv: "c4a1e9f8a87c0c2138850f33f4c2d846"
server_key_version: "encryptionkey@0"
I20260806 18:43:46.139087 2669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:46.156383 2669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:46.159267 2669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:46.159467 2669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:46.160064 2669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:46.160915 2669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:46.160964 2669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:46.161036 2669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:46.161067 2669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:46.171669 2669 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:34233
I20260806 18:43:46.171710 2869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:34233 every 8 connection(s)
I20260806 18:43:46.172611 2669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:43:46.176574 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 2669
I20260806 18:43:46.176697 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:43:46.177049 9159 external_mini_cluster.cc:1428] Setting key 815d964aecc0b09967a1159f30833248
Aug 06 18:43:46 dist-test-slave-k5rr krb5kdc[2498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041826, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:46.186864 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.174507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:46557 (local address 127.8.241.254:41367)
0806 18:43:46.174693 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:46.174698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:46.175469 (+ 771us) server_negotiation.cc:407] Connection header received
0806 18:43:46.176906 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:46.176911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:46.176970 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:46.177083 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:46.178657 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.179422 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.180600 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.180830 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.180920 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.183488 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:46.183492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:46.183495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:46.183523 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:46.184938 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:46.185436 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:46.185439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:46.185440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:46.185513 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:46.185825 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:46.185828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:46.185829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:46.185999 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:46.186086 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:46.186569 (+ 483us) server_negotiation.cc:299] Negotiation successful
0806 18:43:46.186698 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:43:46.187721 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.174853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:41367 (local address 127.8.241.193:46557)
0806 18:43:46.175315 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:46.175361 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:46.176647 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:46.177256 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:46.177264 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:46.177734 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:46.178472 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.178496 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.179563 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.179566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.180484 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.180495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.180974 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.180978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.181188 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.181832 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:46.181848 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:46.183309 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:46.185071 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:46.185077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:46.185092 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:46.185341 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:46.185598 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:46.185601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:46.185603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:46.185738 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:46.186101 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:46.186108 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:46.186345 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 18:43:46.186513 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 18:43:46.186826 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":103,"threads_started":1}
I20260806 18:43:46.188921 2870 heartbeater.cc:347] Connected to a master server at 127.8.241.254:41367
I20260806 18:43:46.189174 2870 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:46.189796 2870 heartbeater.cc:511] Master 127.8.241.254:41367 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:43:46.191326 2573 ts_manager.cc:194] Registered new tserver with Master: 855ba3de6fde42cebc2eae5b3e23392d (127.8.241.193:34233)
I20260806 18:43:46.192485 2573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:46557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:43:46.203850 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:41367
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:46.318770 2876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:46.319099 2876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:46.319157 2876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:46.323158 2876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:46.323212 2876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:46.323273 2876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:43:46.328451 2876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:41367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:46.329516 2876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:46.330499 2876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:46.339978 2886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:46.340016 2889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:46.340037 2887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:46.340296 2876 server_base.cc:1034] running on GCE node
I20260806 18:43:46.340679 2876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:46.341351 2876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:46.342521 2876 hybrid_clock.cc:648] HybridClock initialized: now 1786041826342502 us; error 27 us; skew 500 ppm
Aug 06 18:43:46 dist-test-slave-k5rr krb5kdc[2498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041826, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:46.345327 2876 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:43:46.346145 2876 webserver.cc:460] Webserver started at http://127.8.241.194:34935/ using document root <none> and password file <none>
I20260806 18:43:46.346666 2876 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:46.346724 2876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:46.346894 2876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:46.348867 2876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "a947e06eb83a484a9cbca15f7cf029e4"
format_stamp: "Formatted at 2026-08-06 18:43:46 on dist-test-slave-k5rr"
server_key: "a3bd69b33760d88c5eb6564feca44b8c"
server_key_iv: "420afe03f5cd755a0efd7784c9f6d03a"
server_key_version: "encryptionkey@0"
I20260806 18:43:46.349453 2876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "a947e06eb83a484a9cbca15f7cf029e4"
format_stamp: "Formatted at 2026-08-06 18:43:46 on dist-test-slave-k5rr"
server_key: "a3bd69b33760d88c5eb6564feca44b8c"
server_key_iv: "420afe03f5cd755a0efd7784c9f6d03a"
server_key_version: "encryptionkey@0"
I20260806 18:43:46.353143 2876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:43:46.355507 2899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:46.356753 2876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:46.356873 2876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "a947e06eb83a484a9cbca15f7cf029e4"
format_stamp: "Formatted at 2026-08-06 18:43:46 on dist-test-slave-k5rr"
server_key: "a3bd69b33760d88c5eb6564feca44b8c"
server_key_iv: "420afe03f5cd755a0efd7784c9f6d03a"
server_key_version: "encryptionkey@0"
I20260806 18:43:46.356964 2876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:46.370141 2876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:46.373162 2876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:46.373339 2876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:46.373922 2876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:46.374760 2876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:46.374809 2876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:46.374882 2876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:46.374908 2876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:46.383812 2876 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:43765
I20260806 18:43:46.383834 3068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:43765 every 8 connection(s)
I20260806 18:43:46.384769 2876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:43:46.390264 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 2876
I20260806 18:43:46.390405 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:43:46.390681 9159 external_mini_cluster.cc:1428] Setting key 899743991d4af2a6749c7c65c68e61a6
Aug 06 18:43:46 dist-test-slave-k5rr krb5kdc[2498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041826, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:46.398937 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.386536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:49697 (local address 127.8.241.254:41367)
0806 18:43:46.386695 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:46.386699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:46.387544 (+ 845us) server_negotiation.cc:407] Connection header received
0806 18:43:46.388833 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:46.388837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:46.388885 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:46.388958 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:46.390215 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.390808 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.392161 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.392380 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.392458 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.395323 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:46.395326 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:46.395329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:46.395360 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:46.396741 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:46.397292 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:46.397296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:46.397298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:46.397403 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:46.397761 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:46.397765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:46.397766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:46.397997 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:46.398097 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:46.398630 (+ 533us) server_negotiation.cc:299] Negotiation successful
0806 18:43:46.398766 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:43:46.399708 3071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.386955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:41367 (local address 127.8.241.194:49697)
0806 18:43:46.387399 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:46.387443 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:46.388646 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:46.389155 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:46.389162 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:46.389511 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:46.390052 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.390075 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.390937 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.390941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.392053 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.392063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.392531 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.392536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.392729 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.393324 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:46.393344 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:46.395157 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:46.396847 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:46.396852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:46.396864 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:46.397181 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:46.397514 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:46.397517 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:46.397519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:46.397666 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:46.398084 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:46.398090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:46.398360 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 18:43:46.398533 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 18:43:46.398792 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":104,"threads_started":1}
I20260806 18:43:46.400791 3069 heartbeater.cc:347] Connected to a master server at 127.8.241.254:41367
I20260806 18:43:46.401058 3069 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
I20260806 18:43:46.401593 3069 heartbeater.cc:511] Master 127.8.241.254:41367 requested a full tablet report, sending...
I20260806 18:43:46.402647 2573 ts_manager.cc:194] Registered new tserver with Master: a947e06eb83a484a9cbca15f7cf029e4 (127.8.241.194:43765)
I20260806 18:43:46.403221 2573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:49697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:43:46.417796 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:41367
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:46.531009 3075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:46.531284 3075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:46.531330 3075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:46.535384 3075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:46.535441 3075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:46.535501 3075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:43:46.540925 3075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:41367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:46.542006 3075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:46.543056 3075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:46.552639 3083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:46.552649 3082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:46.552649 3085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:46.552851 3075 server_base.cc:1034] running on GCE node
I20260806 18:43:46.553303 3075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:46.553956 3075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:46.555127 3075 hybrid_clock.cc:648] HybridClock initialized: now 1786041826555108 us; error 31 us; skew 500 ppm
Aug 06 18:43:46 dist-test-slave-k5rr krb5kdc[2498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041826, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:46.557888 3075 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:43:46.558708 3075 webserver.cc:460] Webserver started at http://127.8.241.195:39001/ using document root <none> and password file <none>
I20260806 18:43:46.559237 3075 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:46.559293 3075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:46.559460 3075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:46.561625 3075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "c869c41321f1480cb67991cd7a10f108"
format_stamp: "Formatted at 2026-08-06 18:43:46 on dist-test-slave-k5rr"
server_key: "0a68630850626d78b0c26ed3e46edf70"
server_key_iv: "10134e7c7228b3c42fc4a62172e03ad5"
server_key_version: "encryptionkey@0"
I20260806 18:43:46.562148 3075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "c869c41321f1480cb67991cd7a10f108"
format_stamp: "Formatted at 2026-08-06 18:43:46 on dist-test-slave-k5rr"
server_key: "0a68630850626d78b0c26ed3e46edf70"
server_key_iv: "10134e7c7228b3c42fc4a62172e03ad5"
server_key_version: "encryptionkey@0"
I20260806 18:43:46.565814 3075 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:43:46.568346 3096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:46.569555 3075 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:46.569655 3075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "c869c41321f1480cb67991cd7a10f108"
format_stamp: "Formatted at 2026-08-06 18:43:46 on dist-test-slave-k5rr"
server_key: "0a68630850626d78b0c26ed3e46edf70"
server_key_iv: "10134e7c7228b3c42fc4a62172e03ad5"
server_key_version: "encryptionkey@0"
I20260806 18:43:46.569748 3075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:46.583220 3075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:46.586054 3075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:46.586233 3075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:46.586843 3075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:46.587682 3075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:46.587738 3075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:46.587783 3075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:46.587811 3075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:46.596266 3075 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:39113
I20260806 18:43:46.596288 3279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:39113 every 8 connection(s)
I20260806 18:43:46.597172 3075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:43:46.604476 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 3075
I20260806 18:43:46.604609 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:43:46.604902 9159 external_mini_cluster.cc:1428] Setting key 204249227a4847529ae844f9ce44f55a
Aug 06 18:43:46 dist-test-slave-k5rr krb5kdc[2498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041826, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:46.610946 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.598959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:41947 (local address 127.8.241.254:41367)
0806 18:43:46.599083 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:46.599087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:46.599952 (+ 865us) server_negotiation.cc:407] Connection header received
0806 18:43:46.601299 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:46.601302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:46.601362 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:46.601438 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:46.602593 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.603122 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.604241 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.604458 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.604546 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.607268 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:46.607272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:46.607275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:46.607305 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:46.608760 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:46.609349 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:46.609354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:46.609356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:46.609442 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:46.609803 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:46.609805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:46.609807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:46.610002 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:46.610114 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:46.610633 (+ 519us) server_negotiation.cc:299] Negotiation successful
0806 18:43:46.610766 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:43:46.611747 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.599337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:41367 (local address 127.8.241.195:41947)
0806 18:43:46.599796 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:46.599839 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:46.601099 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:46.601621 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:46.601629 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:46.601971 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:46.602453 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.602467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.603241 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.603244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.604104 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.604114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.604599 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.604603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.604782 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.605442 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:46.605464 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:46.607111 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:46.608889 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:46.608896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:46.608911 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:46.609249 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:46.609533 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:46.609536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:46.609539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:46.609686 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:46.610111 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:46.610118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:46.610380 (+ 262us) client_negotiation.cc:769] Sending connection context
0806 18:43:46.610553 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 18:43:46.610907 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":125,"threads_started":1}
I20260806 18:43:46.612828 3280 heartbeater.cc:347] Connected to a master server at 127.8.241.254:41367
I20260806 18:43:46.613088 3280 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:46.613586 3280 heartbeater.cc:511] Master 127.8.241.254:41367 requested a full tablet report, sending...
I20260806 18:43:46.614589 2573 ts_manager.cc:194] Registered new tserver with Master: c869c41321f1480cb67991cd7a10f108 (127.8.241.195:39113)
I20260806 18:43:46.615136 2573 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:41947
I20260806 18:43:46.619117 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:43:46.627635 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.621348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59352 (local address 127.8.241.254:41367)
0806 18:43:46.621483 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:46.621488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:46.621622 (+ 134us) server_negotiation.cc:407] Connection header received
0806 18:43:46.621717 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:46.621719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:46.621787 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:46.621847 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:46.622475 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.623010 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.623770 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.623969 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.624035 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.625196 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:46.625199 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:46.625201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:46.625223 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:46.626305 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:46.626649 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:46.626652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:46.626653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:46.626715 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:46.626931 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:46.626933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:46.626934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:46.627131 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:46.627226 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:46.627373 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 18:43:46.627503 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:43:46.630474 2573 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:43:46.633499 2573 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:43:46.647799 3304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.642506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:39113 (local address 127.0.0.1:56474)
0806 18:43:46.643372 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:46.643384 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:46.643468 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:46.643907 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:46.643910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:46.643925 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:46.644022 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.644028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.644998 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.645001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.646330 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.646341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.647270 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.647274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.647389 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.647429 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:43:46.647530 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 18:43:46.647644 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260806 18:43:46.647905 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.642729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56474 (local address 127.8.241.195:39113)
0806 18:43:46.643139 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:46.643154 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:46.643554 (+ 400us) server_negotiation.cc:407] Connection header received
0806 18:43:46.643627 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:46.643631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:46.643795 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:46.643929 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:46.644131 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.644872 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.646457 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.647157 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.647255 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.647408 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:46.647643 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 18:43:46.647749 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":127,"threads_started":1}
I20260806 18:43:46.648123 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.641918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.8.241.194:43765)
0806 18:43:46.642295 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:46.642308 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:46.642996 (+ 688us) server_negotiation.cc:407] Connection header received
0806 18:43:46.643088 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:46.643094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:46.643308 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:46.643501 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:46.644026 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.645218 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.646571 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.647302 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.647409 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.647543 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:46.647873 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 18:43:46.647983 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":143,"threads_started":1}
I20260806 18:43:46.647802 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.641747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:43765 (local address 127.0.0.1:38548)
0806 18:43:46.642642 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:46.642692 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:46.642843 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:46.643721 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:46.643728 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:46.643753 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:46.643894 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.643901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.645363 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.645367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.646468 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.646476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.647486 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.647490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.647568 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.647589 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:43:46.647646 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 18:43:46.647696 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":65,"threads_started":1}
I20260806 18:43:46.648625 3306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.642465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57950 (local address 127.8.241.193:34233)
0806 18:43:46.642843 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:46.642878 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:46.643346 (+ 468us) server_negotiation.cc:407] Connection header received
0806 18:43:46.643456 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:46.643461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:46.643643 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:46.643794 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:46.644201 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.645540 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.646812 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.647580 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.647674 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.647791 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:46.648370 (+ 579us) server_negotiation.cc:299] Negotiation successful
0806 18:43:46.648487 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":105,"threads_started":1}
I20260806 18:43:46.648938 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.641844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34233 (local address 127.0.0.1:57950)
0806 18:43:46.643018 (+ 1174us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:46.643030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:46.643241 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:46.643975 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:46.643978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:46.643995 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:46.644098 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.644106 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.645673 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.645676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.646682 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.646694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.647852 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.647856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.647947 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.647971 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:43:46.648040 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:43:46.648109 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1113,"mutex_wait_us":24,"thread_start_us":87,"threads_started":1}
I20260806 18:43:46.650836 3167 tablet_service.cc:1467] Processing CreateTablet for tablet 989da3a9d08d4102950d3475aaaf2035 (DEFAULT_TABLE table=test-table [id=9ce9336441be41f4a1c2ff2b47fafbab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:46.650837 2970 tablet_service.cc:1467] Processing CreateTablet for tablet 989da3a9d08d4102950d3475aaaf2035 (DEFAULT_TABLE table=test-table [id=9ce9336441be41f4a1c2ff2b47fafbab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:46.651890 3167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 989da3a9d08d4102950d3475aaaf2035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:46.651893 2970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 989da3a9d08d4102950d3475aaaf2035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:46.653906 2764 tablet_service.cc:1467] Processing CreateTablet for tablet 989da3a9d08d4102950d3475aaaf2035 (DEFAULT_TABLE table=test-table [id=9ce9336441be41f4a1c2ff2b47fafbab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:46.654851 2764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 989da3a9d08d4102950d3475aaaf2035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:46.657977 3309 tablet_bootstrap.cc:492] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: Bootstrap starting.
I20260806 18:43:46.659379 3311 tablet_bootstrap.cc:492] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: Bootstrap starting.
I20260806 18:43:46.660706 3309 tablet_bootstrap.cc:654] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:46.661599 3312 tablet_bootstrap.cc:492] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d: Bootstrap starting.
I20260806 18:43:46.661721 3311 tablet_bootstrap.cc:654] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:46.662098 3309 log.cc:824] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:46.662969 3311 log.cc:824] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:46.663589 3312 tablet_bootstrap.cc:654] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:46.664151 3309 tablet_bootstrap.cc:492] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: No bootstrap required, opened a new log
I20260806 18:43:46.664363 3309 ts_tablet_manager.cc:1397] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:43:46.664638 3312 log.cc:824] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:46.664950 3311 tablet_bootstrap.cc:492] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: No bootstrap required, opened a new log
I20260806 18:43:46.665220 3311 ts_tablet_manager.cc:1397] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:43:46.666386 3312 tablet_bootstrap.cc:492] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d: No bootstrap required, opened a new log
I20260806 18:43:46.666555 3312 ts_tablet_manager.cc:1397] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:43:46.668524 3309 raft_consensus.cc:348] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } }
I20260806 18:43:46.668813 3309 raft_consensus.cc:374] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:46.668900 3309 raft_consensus.cc:729] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a947e06eb83a484a9cbca15f7cf029e4, State: Initialized, Role: FOLLOWER
I20260806 18:43:46.669111 3311 raft_consensus.cc:348] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } }
I20260806 18:43:46.669373 3311 raft_consensus.cc:374] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:46.669467 3311 raft_consensus.cc:729] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c869c41321f1480cb67991cd7a10f108, State: Initialized, Role: FOLLOWER
I20260806 18:43:46.669397 3309 consensus_queue.cc:260] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } }
I20260806 18:43:46.669401 3312 raft_consensus.cc:348] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } }
I20260806 18:43:46.669636 3312 raft_consensus.cc:374] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:46.669740 3312 raft_consensus.cc:729] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855ba3de6fde42cebc2eae5b3e23392d, State: Initialized, Role: FOLLOWER
I20260806 18:43:46.669917 3311 consensus_queue.cc:260] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } }
I20260806 18:43:46.670156 3312 consensus_queue.cc:260] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } }
I20260806 18:43:46.670406 3309 ts_tablet_manager.cc:1428] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:43:46.670584 3280 heartbeater.cc:503] Master 127.8.241.254:41367 was elected leader, sending a full tablet report...
I20260806 18:43:46.670802 3069 heartbeater.cc:503] Master 127.8.241.254:41367 was elected leader, sending a full tablet report...
I20260806 18:43:46.671749 2870 heartbeater.cc:503] Master 127.8.241.254:41367 was elected leader, sending a full tablet report...
I20260806 18:43:46.672086 3312 ts_tablet_manager.cc:1428] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:43:46.672806 3311 ts_tablet_manager.cc:1428] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260806 18:43:46.674022 2872 tablet.cc:2369] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:43:46.848119 3281 tablet.cc:2369] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:43:46.886070 3070 tablet.cc:2369] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:46.963281 3320 raft_consensus.cc:482] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:46.963521 3320 raft_consensus.cc:504] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } }
I20260806 18:43:46.964715 3320 leader_election.cc:288] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a947e06eb83a484a9cbca15f7cf029e4 (127.8.241.194:43765), c869c41321f1480cb67991cd7a10f108 (127.8.241.195:39113)
I20260806 18:43:46.968729 3322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.965057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:43765 (local address 127.8.241.193:47395)
0806 18:43:46.965491 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:46.965507 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:46.965626 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:46.965884 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:46.965888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:46.965904 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:46.966071 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.966079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.966778 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.966780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.967597 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.967607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.968282 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.968286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.968387 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.968411 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:43:46.968494 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:43:46.968557 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":31,"thread_start_us":140,"threads_started":1}
I20260806 18:43:46.968775 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.965269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:47395 (local address 127.8.241.194:43765)
0806 18:43:46.965428 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:46.965432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:46.965595 (+ 163us) server_negotiation.cc:407] Connection header received
0806 18:43:46.965721 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:46.965723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:46.965779 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:46.965871 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:46.966184 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.966660 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.967719 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.968109 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.968200 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.968243 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:46.968512 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 18:43:46.968647 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:43:46.968932 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.965170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:46821 (local address 127.8.241.195:39113)
0806 18:43:46.965308 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:46.965312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:46.965474 (+ 162us) server_negotiation.cc:407] Connection header received
0806 18:43:46.965614 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:46.965618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:46.965671 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:46.965765 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:46.966073 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.966616 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.967688 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.968233 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.968368 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.968421 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:46.968690 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 18:43:46.968824 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:43:46.969285 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:46.965035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:39113 (local address 127.8.241.193:46821)
0806 18:43:46.965349 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:46.965387 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:46.965504 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:46.965774 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:46.965777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:46.965793 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:46.965959 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.965967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.966728 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.966732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.967563 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:46.967572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:46.968375 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:46.968378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:46.968460 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:46.968491 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 18:43:46.969076 (+ 585us) client_negotiation.cc:240] Negotiation successful
0806 18:43:46.969150 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":7,"thread_start_us":88,"threads_started":1}
I20260806 18:43:46.969506 3002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989da3a9d08d4102950d3475aaaf2035" candidate_uuid: "855ba3de6fde42cebc2eae5b3e23392d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a947e06eb83a484a9cbca15f7cf029e4" is_pre_election: true
I20260806 18:43:46.969836 3203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989da3a9d08d4102950d3475aaaf2035" candidate_uuid: "855ba3de6fde42cebc2eae5b3e23392d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c869c41321f1480cb67991cd7a10f108" is_pre_election: true
I20260806 18:43:46.969870 3002 raft_consensus.cc:2454] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 855ba3de6fde42cebc2eae5b3e23392d in term 0.
I20260806 18:43:46.970094 3203 raft_consensus.cc:2454] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 855ba3de6fde42cebc2eae5b3e23392d in term 0.
I20260806 18:43:46.970346 2697 leader_election.cc:302] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 855ba3de6fde42cebc2eae5b3e23392d, a947e06eb83a484a9cbca15f7cf029e4; no voters:
I20260806 18:43:46.970614 3320 raft_consensus.cc:2781] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:43:46.970693 3320 raft_consensus.cc:482] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:43:46.970724 3320 raft_consensus.cc:3037] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:46.971743 3320 raft_consensus.cc:504] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } }
I20260806 18:43:46.972076 3320 leader_election.cc:288] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [CANDIDATE]: Term 1 election: Requested vote from peers a947e06eb83a484a9cbca15f7cf029e4 (127.8.241.194:43765), c869c41321f1480cb67991cd7a10f108 (127.8.241.195:39113)
I20260806 18:43:46.972422 3002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989da3a9d08d4102950d3475aaaf2035" candidate_uuid: "855ba3de6fde42cebc2eae5b3e23392d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a947e06eb83a484a9cbca15f7cf029e4"
I20260806 18:43:46.972426 3203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "989da3a9d08d4102950d3475aaaf2035" candidate_uuid: "855ba3de6fde42cebc2eae5b3e23392d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c869c41321f1480cb67991cd7a10f108"
I20260806 18:43:46.972536 3203 raft_consensus.cc:3037] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:46.972548 3002 raft_consensus.cc:3037] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:46.973685 3203 raft_consensus.cc:2454] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 855ba3de6fde42cebc2eae5b3e23392d in term 1.
I20260806 18:43:46.973697 3002 raft_consensus.cc:2454] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 855ba3de6fde42cebc2eae5b3e23392d in term 1.
I20260806 18:43:46.973973 2696 leader_election.cc:302] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 855ba3de6fde42cebc2eae5b3e23392d, c869c41321f1480cb67991cd7a10f108; no voters:
I20260806 18:43:46.974139 3320 raft_consensus.cc:2781] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:46.974387 3320 raft_consensus.cc:686] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 1 LEADER]: Becoming Leader. State: Replica: 855ba3de6fde42cebc2eae5b3e23392d, State: Running, Role: LEADER
I20260806 18:43:46.974763 3320 consensus_queue.cc:237] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } }
I20260806 18:43:46.977841 2570 catalog_manager.cc:5495] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d reported cstate change: term changed from 0 to 1, leader changed from <none> to 855ba3de6fde42cebc2eae5b3e23392d (127.8.241.193). New cstate: current_term: 1 leader_uuid: "855ba3de6fde42cebc2eae5b3e23392d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "855ba3de6fde42cebc2eae5b3e23392d" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 } health_report { overall_health: UNKNOWN } } }
I20260806 18:43:47.013119 3306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.009839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.8.241.193:34233)
0806 18:43:47.009960 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:47.009963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:47.009976 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:43:47.010105 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:47.010107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:47.010165 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:47.010269 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:43:47.010459 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.010951 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.011798 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.012019 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.012097 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:47.012329 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:43:47.012776 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:43:47.012867 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:47.012911 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 18:43:47.012965 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:43:47.019090 3002 raft_consensus.cc:1261] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [term 1 FOLLOWER]: Refusing update from remote peer 855ba3de6fde42cebc2eae5b3e23392d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:47.019090 3203 raft_consensus.cc:1261] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [term 1 FOLLOWER]: Refusing update from remote peer 855ba3de6fde42cebc2eae5b3e23392d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:47.019824 3323 consensus_queue.cc:1035] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c869c41321f1480cb67991cd7a10f108" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 39113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:47.020012 3320 consensus_queue.cc:1035] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a947e06eb83a484a9cbca15f7cf029e4" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 43765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:47.028779 3333 mvcc.cc:205] Tried to move back new op lower bound from 7315627323462942720 to 7315627323291062272. Current Snapshot: MvccSnapshot[applied={T|T < 7315627323462942720}]
I20260806 18:43:47.037159 2570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:47.037361 2570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:47.039842 2570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2eee1fcf60e94f51848d46d64b9e95f6: Sending DeleteTablet for 3 replicas of tablet 989da3a9d08d4102950d3475aaaf2035
I20260806 18:43:47.040576 2764 tablet_service.cc:1514] Processing DeleteTablet for tablet 989da3a9d08d4102950d3475aaaf2035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:47 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:57950
I20260806 18:43:47.040576 3167 tablet_service.cc:1514] Processing DeleteTablet for tablet 989da3a9d08d4102950d3475aaaf2035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:47 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:56474
I20260806 18:43:47.040587 2970 tablet_service.cc:1514] Processing DeleteTablet for tablet 989da3a9d08d4102950d3475aaaf2035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:47 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:38548
I20260806 18:43:47.041091 3346 tablet_replica.cc:330] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: stopping tablet replica
I20260806 18:43:47.041227 3343 tablet_replica.cc:330] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d: stopping tablet replica
I20260806 18:43:47.041473 3346 raft_consensus.cc:2229] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:47.041610 3343 raft_consensus.cc:2229] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:43:47.041632 3345 tablet_replica.cc:330] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: stopping tablet replica
I20260806 18:43:47.041843 3346 raft_consensus.cc:2258] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:47.042042 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 2669
I20260806 18:43:47.042049 3343 raft_consensus.cc:2258] T 989da3a9d08d4102950d3475aaaf2035 P 855ba3de6fde42cebc2eae5b3e23392d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:47.048897 3345 raft_consensus.cc:2229] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:47.049271 3345 raft_consensus.cc:2258] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:47.051113 3345 ts_tablet_manager.cc:1905] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:47.051184 3346 ts_tablet_manager.cc:1905] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:47.053932 3345 ts_tablet_manager.cc:1918] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:47.053932 3346 ts_tablet_manager.cc:1918] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:47.054041 3345 log.cc:1196] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/989da3a9d08d4102950d3475aaaf2035
I20260806 18:43:47.054041 3346 log.cc:1196] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/989da3a9d08d4102950d3475aaaf2035
I20260806 18:43:47.054373 3345 ts_tablet_manager.cc:1939] T 989da3a9d08d4102950d3475aaaf2035 P c869c41321f1480cb67991cd7a10f108: Deleting consensus metadata
I20260806 18:43:47.054379 3346 ts_tablet_manager.cc:1939] T 989da3a9d08d4102950d3475aaaf2035 P a947e06eb83a484a9cbca15f7cf029e4: Deleting consensus metadata
I20260806 18:43:47.055734 2548 catalog_manager.cc:4841] TS a947e06eb83a484a9cbca15f7cf029e4 (127.8.241.194:43765): tablet 989da3a9d08d4102950d3475aaaf2035 (table test-table [id=9ce9336441be41f4a1c2ff2b47fafbab]) successfully deleted
I20260806 18:43:47.055806 2547 catalog_manager.cc:4841] TS c869c41321f1480cb67991cd7a10f108 (127.8.241.195:39113): tablet 989da3a9d08d4102950d3475aaaf2035 (table test-table [id=9ce9336441be41f4a1c2ff2b47fafbab]) successfully deleted
W20260806 18:43:47.056823 2545 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:34233 (error 108)
I20260806 18:43:47.057641 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 2876
I20260806 18:43:47.057855 3304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.057570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34233 (local address 127.0.0.1:57972)
0806 18:43:47.057693 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:47.057764 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:34233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:43:47.065168 2545 catalog_manager.cc:4584] TS 855ba3de6fde42cebc2eae5b3e23392d (127.8.241.193:34233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 989da3a9d08d4102950d3475aaaf2035: Network error: Client connection negotiation failed: client connection to 127.8.241.193:34233: connect: Connection refused (error 111)
I20260806 18:43:47.067703 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 3075
I20260806 18:43:47.099056 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 2514
2026-08-06T18:43:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2624 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T18:43:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:43:47Z Disabled control of system clock
I20260806 18:43:47.131238 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:43473
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:43473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:43:47.246291 3351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:47.246562 3351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:47.246598 3351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:47.250641 3351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:47.250695 3351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:47.250710 3351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:47.250725 3351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:47.250737 3351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:47.255446 3351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:43473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:43473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:47.256507 3351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:47.257977 3351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:47.266413 3360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:47.266390 3363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:47.266388 3361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:47.266597 3351 server_base.cc:1034] running on GCE node
I20260806 18:43:47.267333 3351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:47.268549 3351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:47.269704 3351 hybrid_clock.cc:648] HybridClock initialized: now 1786041827269686 us; error 41 us; skew 500 ppm
I20260806 18:43:47.271649 3351 webserver.cc:460] Webserver started at http://127.8.241.254:45605/ using document root <none> and password file <none>
I20260806 18:43:47.272236 3351 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:47.272308 3351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:47.272537 3351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:47.274703 3351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "3be9954f41a94668b7980c256db57153"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "8f4243dc26a552308ad8b5ceee39ddf0"
server_key_iv: "90c9c5cddbdf4e36ad5dfa277da5d961"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.275265 3351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "3be9954f41a94668b7980c256db57153"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "8f4243dc26a552308ad8b5ceee39ddf0"
server_key_iv: "90c9c5cddbdf4e36ad5dfa277da5d961"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.279048 3351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:43:47.281611 3375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.282853 3351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:47.282974 3351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "3be9954f41a94668b7980c256db57153"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "8f4243dc26a552308ad8b5ceee39ddf0"
server_key_iv: "90c9c5cddbdf4e36ad5dfa277da5d961"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.283074 3351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:47.312546 3351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:47.313328 3351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:47.313519 3351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:47.320142 3351 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:43473
I20260806 18:43:47.320156 3459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:43473 every 8 connection(s)
I20260806 18:43:47.321122 3351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:47.324272 3462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:47.327041 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 3351
I20260806 18:43:47.327171 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:43:47.327500 9159 external_mini_cluster.cc:1428] Setting key a56869f60c8f781aa0f29fe4c413f7da
I20260806 18:43:47.330705 3462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153: Bootstrap starting.
I20260806 18:43:47.332991 3462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:47.333796 3462 log.cc:824] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:47.336018 3462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153: No bootstrap required, opened a new log
I20260806 18:43:47.337157 3466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.329119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54114 (local address 127.8.241.254:43473)
0806 18:43:47.329789 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:47.329804 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:47.329839 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:43:47.330485 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:47.330503 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:47.330933 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:47.331244 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:47.331718 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.332666 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.333589 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.334102 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.334264 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:47.335046 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:47.335053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:47.335069 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:47.335563 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:47.335638 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:47.335667 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:47.335766 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:47.335895 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 18:43:47.336183 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":135,"threads_started":1}
I20260806 18:43:47.340164 3462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9954f41a94668b7980c256db57153" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43473 } }
I20260806 18:43:47.340390 3462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:47.340430 3462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3be9954f41a94668b7980c256db57153, State: Initialized, Role: FOLLOWER
I20260806 18:43:47.340787 3462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9954f41a94668b7980c256db57153" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43473 } }
I20260806 18:43:47.340899 3462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:47.340943 3462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:47.341040 3462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:47.341874 3462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9954f41a94668b7980c256db57153" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43473 } }
I20260806 18:43:47.342211 3462 leader_election.cc:302] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3be9954f41a94668b7980c256db57153; no voters:
I20260806 18:43:47.342535 3462 leader_election.cc:288] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:43:47.342640 3469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:47.342952 3469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [term 1 LEADER]: Becoming Leader. State: Replica: 3be9954f41a94668b7980c256db57153, State: Running, Role: LEADER
I20260806 18:43:47.343241 3469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9954f41a94668b7980c256db57153" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43473 } }
I20260806 18:43:47.343629 3462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:47.344887 3474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3be9954f41a94668b7980c256db57153" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9954f41a94668b7980c256db57153" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43473 } } }
I20260806 18:43:47.344915 3476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3be9954f41a94668b7980c256db57153. Latest consensus state: current_term: 1 leader_uuid: "3be9954f41a94668b7980c256db57153" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9954f41a94668b7980c256db57153" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 43473 } } }
I20260806 18:43:47.345127 3474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:47.345145 3476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153 [sys.catalog]: This master's current role is: LEADER
W20260806 18:43:47.348459 3494 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:43:47.348541 3494 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:43:47.348615 3491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:47.349377 3491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:47.354864 3491 catalog_manager.cc:1347] Generated new cluster ID: 24ba64e8c23d4d5cae91f6926b728bcc
I20260806 18:43:47.354923 3491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:47.361091 3491 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:43:47.362234 3491 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:47.370822 3491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153: Generated new TSK 0
I20260806 18:43:47.371634 3491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:43:47.378926 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43473
--builtin_ntp_servers=127.8.241.212:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:43:47.496449 3498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:47.496704 3498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:47.496752 3498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:47.500725 3498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:47.500782 3498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:47.500838 3498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:43:47.505563 3498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:47.506618 3498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:47.508023 3498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:47.517601 3506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:47.517608 3508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:47.517755 3498 server_base.cc:1034] running on GCE node
W20260806 18:43:47.517608 3510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:47.518265 3498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:47.518893 3498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:47.520073 3498 hybrid_clock.cc:648] HybridClock initialized: now 1786041827520053 us; error 29 us; skew 500 ppm
I20260806 18:43:47.521951 3498 webserver.cc:460] Webserver started at http://127.8.241.193:46251/ using document root <none> and password file <none>
I20260806 18:43:47.522541 3498 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:47.522612 3498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:47.522831 3498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:47.524886 3498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "bdd6430138eb4307b068c24acbb3bade"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "736bda7b8340215e73bc06ae282906f4"
server_key_iv: "bb52281619c22f7ba86050f6e117604e"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.525496 3498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "bdd6430138eb4307b068c24acbb3bade"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "736bda7b8340215e73bc06ae282906f4"
server_key_iv: "bb52281619c22f7ba86050f6e117604e"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.529094 3498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:43:47.531435 3519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.532601 3498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:47.532709 3498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "bdd6430138eb4307b068c24acbb3bade"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "736bda7b8340215e73bc06ae282906f4"
server_key_iv: "bb52281619c22f7ba86050f6e117604e"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.532833 3498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:47.560334 3498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:47.561146 3498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:47.561331 3498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:47.561960 3498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:47.562968 3498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:47.563024 3498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.563082 3498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:47.563107 3498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.572314 3498 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:33591
I20260806 18:43:47.572355 3690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:33591 every 8 connection(s)
I20260806 18:43:47.573254 3498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:43:47.576426 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 3498
I20260806 18:43:47.576550 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:43:47.576970 9159 external_mini_cluster.cc:1428] Setting key 5941f051a96a0b7459962c8402032cde
I20260806 18:43:47.579190 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43473
--builtin_ntp_servers=127.8.241.212:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:43:47.583331 3466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.575080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:58625 (local address 127.8.241.254:43473)
0806 18:43:47.575241 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:47.575247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:47.576487 (+ 1240us) server_negotiation.cc:407] Connection header received
0806 18:43:47.578243 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:47.578246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:47.578302 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:47.578379 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:47.579584 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.580116 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.581508 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.581726 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.581824 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:47.582875 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:47.582878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:47.582880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:47.582893 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:47.582911 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:47.582918 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:47.582977 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:47.583123 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 18:43:47.583186 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:43:47.584192 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.575520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43473 (local address 127.8.241.193:58625)
0806 18:43:47.576307 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:47.576369 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:47.578030 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:47.578586 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:47.578595 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:47.578704 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:47.579432 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:47.579455 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.580255 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.580259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:47.581390 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:47.581400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.581883 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.581888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:47.582042 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:47.582613 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:43:47.582614 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:47.582656 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:43:47.582725 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:47.582988 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:47.582997 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:47.583004 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:43:47.583115 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 18:43:47.583414 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
I20260806 18:43:47.585422 3691 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43473
I20260806 18:43:47.585654 3691 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:47.586228 3691 heartbeater.cc:511] Master 127.8.241.254:43473 requested a full tablet report, sending...
I20260806 18:43:47.587806 3397 ts_manager.cc:194] Registered new tserver with Master: bdd6430138eb4307b068c24acbb3bade (127.8.241.193:33591)
I20260806 18:43:47.589156 3397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.193:58625
W20260806 18:43:47.694062 3695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:47.694344 3695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:47.694398 3695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:47.698494 3695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:47.698555 3695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:47.698632 3695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:43:47.703410 3695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:47.704517 3695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:47.706018 3695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:47.715674 3705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:47.715694 3701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:47.715679 3703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:47.715938 3695 server_base.cc:1034] running on GCE node
I20260806 18:43:47.716467 3695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:47.717137 3695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:47.718282 3695 hybrid_clock.cc:648] HybridClock initialized: now 1786041827718255 us; error 42 us; skew 500 ppm
I20260806 18:43:47.720144 3695 webserver.cc:460] Webserver started at http://127.8.241.194:44873/ using document root <none> and password file <none>
I20260806 18:43:47.720717 3695 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:47.720793 3695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:47.720996 3695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:47.723141 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "fb6eece75185439d977d3a91942e8dfa"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "b7957f18cf14413ad164e14048249065"
server_key_iv: "508a50fbec7667dc49977aaea0c581e4"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.723660 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "fb6eece75185439d977d3a91942e8dfa"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "b7957f18cf14413ad164e14048249065"
server_key_iv: "508a50fbec7667dc49977aaea0c581e4"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.727195 3695 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260806 18:43:47.729715 3717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.731055 3695 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:47.731177 3695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "fb6eece75185439d977d3a91942e8dfa"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "b7957f18cf14413ad164e14048249065"
server_key_iv: "508a50fbec7667dc49977aaea0c581e4"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.731300 3695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:47.756011 3695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:47.756773 3695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:47.756958 3695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:47.757603 3695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:47.758592 3695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:47.758641 3695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.758697 3695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:47.758725 3695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.766983 3695 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:46665
I20260806 18:43:47.767022 3895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:46665 every 8 connection(s)
I20260806 18:43:47.767897 3695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:43:47.776360 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 3695
I20260806 18:43:47.776476 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:43:47.776782 9159 external_mini_cluster.cc:1428] Setting key 9dbf5532e53e6b10fb4ecb6a620eba4f
I20260806 18:43:47.777484 3466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.769730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:49475 (local address 127.8.241.254:43473)
0806 18:43:47.769882 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:47.769886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:47.771073 (+ 1187us) server_negotiation.cc:407] Connection header received
0806 18:43:47.772557 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:47.772560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:47.772609 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:47.772685 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:47.773674 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.774259 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.775407 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.775626 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.775708 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:47.776865 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:47.776869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:47.776872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:47.776887 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:47.776914 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:47.776927 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:47.777001 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:47.777250 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 18:43:47.777319 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:43:47.778364 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.770124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43473 (local address 127.8.241.194:49475)
0806 18:43:47.770870 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:47.770930 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:47.772336 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:47.772875 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:47.772882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:47.772968 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:47.773527 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:47.773542 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.774388 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.774392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:47.775282 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:47.775292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.775798 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.775801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:47.775943 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:47.776583 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:43:47.776585 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:47.776633 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:43:47.776697 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:47.777043 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:47.777054 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:47.777061 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:43:47.777233 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 18:43:47.777509 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":120,"threads_started":1}
I20260806 18:43:47.778779 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43473
--builtin_ntp_servers=127.8.241.212:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:43:47.779458 3896 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43473
I20260806 18:43:47.779737 3896 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:47.780386 3896 heartbeater.cc:511] Master 127.8.241.254:43473 requested a full tablet report, sending...
I20260806 18:43:47.781558 3397 ts_manager.cc:194] Registered new tserver with Master: fb6eece75185439d977d3a91942e8dfa (127.8.241.194:46665)
I20260806 18:43:47.782265 3397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.194:49475
W20260806 18:43:47.893582 3900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:47.893841 3900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:47.893878 3900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:47.897735 3900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:47.897781 3900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:47.897832 3900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:43:47.902386 3900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:43473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:47.903378 3900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:47.904699 3900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:43:47.914156 3900 server_base.cc:1034] running on GCE node
W20260806 18:43:47.914124 3909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:47.914103 3914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:47.914094 3911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:47.914839 3900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:47.915445 3900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:47.916604 3900 hybrid_clock.cc:648] HybridClock initialized: now 1786041827916575 us; error 43 us; skew 500 ppm
I20260806 18:43:47.921530 3900 webserver.cc:460] Webserver started at http://127.8.241.195:34591/ using document root <none> and password file <none>
I20260806 18:43:47.922124 3900 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:47.922195 3900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:47.922422 3900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:47.924492 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "7e5b6d295f7a400f8087e50764e40ddd"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "88c14238b22b1d24e330c8e739837ef0"
server_key_iv: "8ddb2c8429d395a075e669aaf7994b93"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.925074 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "7e5b6d295f7a400f8087e50764e40ddd"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "88c14238b22b1d24e330c8e739837ef0"
server_key_iv: "8ddb2c8429d395a075e669aaf7994b93"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.928943 3900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260806 18:43:47.931324 3922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.932472 3900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:43:47.932581 3900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "7e5b6d295f7a400f8087e50764e40ddd"
format_stamp: "Formatted at 2026-08-06 18:43:47 on dist-test-slave-k5rr"
server_key: "88c14238b22b1d24e330c8e739837ef0"
server_key_iv: "8ddb2c8429d395a075e669aaf7994b93"
server_key_version: "encryptionkey@0"
I20260806 18:43:47.932673 3900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:47.951543 3900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:47.952206 3900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:47.952390 3900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:47.952975 3900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:47.953948 3900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:47.953996 3900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.954048 3900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:47.954077 3900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:47.962385 3900 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:45129
I20260806 18:43:47.962428 4095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:45129 every 8 connection(s)
I20260806 18:43:47.963253 3900 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:43:47.965749 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 3900
I20260806 18:43:47.965860 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:43:47.966144 9159 external_mini_cluster.cc:1428] Setting key a2eb68129801370ec91ae2cd13a954da
I20260806 18:43:47.972199 3466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.964993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:44977 (local address 127.8.241.254:43473)
0806 18:43:47.965124 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:47.965128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:47.966200 (+ 1072us) server_negotiation.cc:407] Connection header received
0806 18:43:47.967621 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:47.967624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:47.967680 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:47.967762 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:47.968664 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.969268 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.970374 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.970599 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.970686 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:47.971745 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:47.971748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:47.971751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:47.971764 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:47.971784 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:47.971792 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:47.971850 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:47.972000 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 18:43:47.972060 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:43:47.973037 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.965379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:43473 (local address 127.8.241.195:44977)
0806 18:43:47.966057 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:47.966097 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:47.967418 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:47.967953 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:47.967960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:47.968043 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:47.968526 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:47.968540 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.969392 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.969395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:47.970256 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:47.970266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.970734 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.970738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:47.970899 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:47.971491 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:43:47.971492 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:47.971531 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:43:47.971592 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:47.971861 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:47.971871 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:47.971877 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:43:47.971980 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 18:43:47.972259 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":126,"threads_started":1}
I20260806 18:43:47.974056 4096 heartbeater.cc:347] Connected to a master server at 127.8.241.254:43473
I20260806 18:43:47.974270 4096 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:47.974751 4096 heartbeater.cc:511] Master 127.8.241.254:43473 requested a full tablet report, sending...
I20260806 18:43:47.975729 3397 ts_manager.cc:194] Registered new tserver with Master: 7e5b6d295f7a400f8087e50764e40ddd (127.8.241.195:45129)
I20260806 18:43:47.976253 3397 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.195:44977
I20260806 18:43:47.979732 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:43:47.985380 3466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.981725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.8.241.254:43473)
0806 18:43:47.981859 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:47.981864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:47.982030 (+ 166us) server_negotiation.cc:407] Connection header received
0806 18:43:47.982165 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:47.982168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:47.982213 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:47.982274 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:47.982552 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.983076 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.983881 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:47.984106 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:47.984192 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:47.984961 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:47.984964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:47.984967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:47.984980 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:47.984999 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:47.985041 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:47.985102 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:47.985189 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 18:43:47.985246 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:43:47.988194 3397 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:43:47.991410 3397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:43:48.005316 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.999547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:45129 (local address 127.0.0.1:52396)
0806 18:43:48.000083 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:48.000129 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:48.000271 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:48.001041 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:48.001047 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:48.001072 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:48.001199 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.001207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.002477 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.002480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.003744 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.003756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.004770 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.004774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.004878 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.004917 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:43:48.005049 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 18:43:48.005166 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":56,"thread_start_us":83,"threads_started":1}
I20260806 18:43:48.005343 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.999901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52396 (local address 127.8.241.195:45129)
0806 18:43:48.000340 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:48.000353 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:48.000381 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:43:48.000460 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:48.000466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:48.000682 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:48.000836 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:48.001346 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.002362 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.003883 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.004676 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.004778 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.004986 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:48.005067 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 18:43:48.005165 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":100,"threads_started":1}
I20260806 18:43:48.006022 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:47.999680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:46665 (local address 127.0.0.1:35796)
0806 18:43:48.000586 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:48.000598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:48.000677 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:48.001678 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:48.001681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:48.001699 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:48.001773 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.001778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.003217 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.003221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.004420 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.004431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.005734 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.005737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.005801 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.005819 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:43:48.005870 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 18:43:48.005919 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":3,"thread_start_us":139,"threads_started":1}
I20260806 18:43:48.006207 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:48.000281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:33591 (local address 127.0.0.1:47966)
0806 18:43:48.001906 (+ 1625us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:48.001919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:48.002010 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:48.002416 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:48.002419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:48.002438 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:48.002596 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.002604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.003814 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.003818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.004568 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.004579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.005821 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.005825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.005913 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.005936 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:43:48.006012 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:43:48.006081 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1561,"mutex_wait_us":30,"thread_start_us":86,"threads_started":1}
I20260806 18:43:48.006345 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:48.001028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47966 (local address 127.8.241.193:33591)
0806 18:43:48.001338 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:48.001349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:48.002015 (+ 666us) server_negotiation.cc:407] Connection header received
0806 18:43:48.002103 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:48.002108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:48.002278 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:48.002428 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:48.002703 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.003726 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.004698 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.005465 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.005556 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.005752 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:48.006080 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 18:43:48.006182 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":107,"threads_started":1}
I20260806 18:43:48.006717 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:48.000581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35796 (local address 127.8.241.194:46665)
0806 18:43:48.001196 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:48.001207 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:48.001231 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:43:48.001316 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:48.001321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:48.001524 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:48.001675 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:48.001907 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.002970 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.004552 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.005333 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.005437 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.006414 (+ 977us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:48.006469 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:43:48.006559 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":182,"threads_started":1}
I20260806 18:43:48.008786 4001 tablet_service.cc:1467] Processing CreateTablet for tablet e8dc6f1b68b7473d8d89667508248d79 (DEFAULT_TABLE table=test-table [id=c35ec0ef63294003b3f69eb26b99d056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:48.008920 3786 tablet_service.cc:1467] Processing CreateTablet for tablet e8dc6f1b68b7473d8d89667508248d79 (DEFAULT_TABLE table=test-table [id=c35ec0ef63294003b3f69eb26b99d056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:48.009348 3588 tablet_service.cc:1467] Processing CreateTablet for tablet e8dc6f1b68b7473d8d89667508248d79 (DEFAULT_TABLE table=test-table [id=c35ec0ef63294003b3f69eb26b99d056]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:48.009984 3786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8dc6f1b68b7473d8d89667508248d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:48.009989 4001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8dc6f1b68b7473d8d89667508248d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:48.010257 3588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8dc6f1b68b7473d8d89667508248d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:48.016054 4124 tablet_bootstrap.cc:492] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: Bootstrap starting.
I20260806 18:43:48.017599 4127 tablet_bootstrap.cc:492] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: Bootstrap starting.
I20260806 18:43:48.018021 4125 tablet_bootstrap.cc:492] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade: Bootstrap starting.
I20260806 18:43:48.018602 4124 tablet_bootstrap.cc:654] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:48.019306 4124 log.cc:824] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:48.019896 4125 tablet_bootstrap.cc:654] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:48.020771 4125 log.cc:824] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:48.021440 4127 tablet_bootstrap.cc:654] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:48.021944 4124 tablet_bootstrap.cc:492] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: No bootstrap required, opened a new log
I20260806 18:43:48.022151 4127 log.cc:824] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:48.022166 4124 ts_tablet_manager.cc:1397] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:43:48.023367 4125 tablet_bootstrap.cc:492] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade: No bootstrap required, opened a new log
I20260806 18:43:48.023614 4125 ts_tablet_manager.cc:1397] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:43:48.023874 4127 tablet_bootstrap.cc:492] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: No bootstrap required, opened a new log
I20260806 18:43:48.024109 4127 ts_tablet_manager.cc:1397] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:43:48.026459 4124 raft_consensus.cc:348] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } }
I20260806 18:43:48.026747 4124 raft_consensus.cc:374] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:48.026830 4124 raft_consensus.cc:729] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb6eece75185439d977d3a91942e8dfa, State: Initialized, Role: FOLLOWER
I20260806 18:43:48.027187 4127 raft_consensus.cc:348] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } }
I20260806 18:43:48.027387 4127 raft_consensus.cc:374] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:48.027307 4124 consensus_queue.cc:260] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } }
I20260806 18:43:48.027462 4127 raft_consensus.cc:729] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e5b6d295f7a400f8087e50764e40ddd, State: Initialized, Role: FOLLOWER
I20260806 18:43:48.027419 4125 raft_consensus.cc:348] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } }
I20260806 18:43:48.027731 4125 raft_consensus.cc:374] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:48.027825 4125 raft_consensus.cc:729] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd6430138eb4307b068c24acbb3bade, State: Initialized, Role: FOLLOWER
I20260806 18:43:48.027994 3896 heartbeater.cc:503] Master 127.8.241.254:43473 was elected leader, sending a full tablet report...
I20260806 18:43:48.027928 4127 consensus_queue.cc:260] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } }
I20260806 18:43:48.028445 4124 ts_tablet_manager.cc:1428] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:43:48.028375 4125 consensus_queue.cc:260] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } }
I20260806 18:43:48.028827 4127 ts_tablet_manager.cc:1428] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:43:48.029192 4096 heartbeater.cc:503] Master 127.8.241.254:43473 was elected leader, sending a full tablet report...
I20260806 18:43:48.029296 4125 ts_tablet_manager.cc:1428] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:43:48.029615 3691 heartbeater.cc:503] Master 127.8.241.254:43473 was elected leader, sending a full tablet report...
I20260806 18:43:48.052779 4132 raft_consensus.cc:482] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:48.052979 4132 raft_consensus.cc:504] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } }
I20260806 18:43:48.054204 4132 leader_election.cc:288] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e5b6d295f7a400f8087e50764e40ddd (127.8.241.195:45129), bdd6430138eb4307b068c24acbb3bade (127.8.241.193:33591)
I20260806 18:43:48.057655 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:48.054414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:45129 (local address 127.8.241.194:52579)
0806 18:43:48.054547 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:48.054563 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:48.054669 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:48.054987 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:48.054990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:48.055010 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:48.055120 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.055126 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.055830 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.055834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.056614 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.056623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.057262 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.057266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.057364 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.057384 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:43:48.057449 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:43:48.057521 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 18:43:48.057655 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:48.054547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:52579 (local address 127.8.241.195:45129)
0806 18:43:48.054695 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:48.054701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:48.054716 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:43:48.054821 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:48.054824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:48.054882 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:48.054959 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:48.055229 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.055725 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.056741 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.057157 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.057262 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.057318 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:48.057472 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 18:43:48.057537 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:43:48.058015 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:48.054810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:50651 (local address 127.8.241.193:33591)
0806 18:43:48.054939 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:48.054943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:48.055094 (+ 151us) server_negotiation.cc:407] Connection header received
0806 18:43:48.055198 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:48.055201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:48.055254 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:48.055315 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:48.055569 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.056051 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.057064 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.057450 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.057553 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.057600 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:48.057815 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 18:43:48.057897 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:43:48.058365 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:48.054661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:33591 (local address 127.8.241.194:50651)
0806 18:43:48.055005 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:48.055015 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:48.055108 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:48.055336 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:48.055338 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:48.055354 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:48.055466 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.055471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.056163 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.056166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.056922 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:48.056931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.057564 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.057568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:48.057657 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.057675 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:43:48.058159 (+ 484us) client_negotiation.cc:240] Negotiation successful
0806 18:43:48.058229 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":136,"threads_started":1}
I20260806 18:43:48.058373 4025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8dc6f1b68b7473d8d89667508248d79" candidate_uuid: "fb6eece75185439d977d3a91942e8dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e5b6d295f7a400f8087e50764e40ddd" is_pre_election: true
I20260806 18:43:48.058684 4025 raft_consensus.cc:2454] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb6eece75185439d977d3a91942e8dfa in term 0.
I20260806 18:43:48.059044 3620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8dc6f1b68b7473d8d89667508248d79" candidate_uuid: "fb6eece75185439d977d3a91942e8dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdd6430138eb4307b068c24acbb3bade" is_pre_election: true
I20260806 18:43:48.059172 3720 leader_election.cc:302] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e5b6d295f7a400f8087e50764e40ddd, fb6eece75185439d977d3a91942e8dfa; no voters:
I20260806 18:43:48.059350 3620 raft_consensus.cc:2454] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb6eece75185439d977d3a91942e8dfa in term 0.
I20260806 18:43:48.059453 4132 raft_consensus.cc:2781] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:43:48.059530 4132 raft_consensus.cc:482] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:43:48.059572 4132 raft_consensus.cc:3037] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:48.060595 4132 raft_consensus.cc:504] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } }
I20260806 18:43:48.060906 4132 leader_election.cc:288] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [CANDIDATE]: Term 1 election: Requested vote from peers 7e5b6d295f7a400f8087e50764e40ddd (127.8.241.195:45129), bdd6430138eb4307b068c24acbb3bade (127.8.241.193:33591)
I20260806 18:43:48.061270 4025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8dc6f1b68b7473d8d89667508248d79" candidate_uuid: "fb6eece75185439d977d3a91942e8dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e5b6d295f7a400f8087e50764e40ddd"
I20260806 18:43:48.061319 3620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8dc6f1b68b7473d8d89667508248d79" candidate_uuid: "fb6eece75185439d977d3a91942e8dfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdd6430138eb4307b068c24acbb3bade"
I20260806 18:43:48.061395 4025 raft_consensus.cc:3037] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:48.061431 3620 raft_consensus.cc:3037] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:48.062440 4025 raft_consensus.cc:2454] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb6eece75185439d977d3a91942e8dfa in term 1.
I20260806 18:43:48.062517 3620 raft_consensus.cc:2454] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb6eece75185439d977d3a91942e8dfa in term 1.
I20260806 18:43:48.062779 3720 leader_election.cc:302] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e5b6d295f7a400f8087e50764e40ddd, fb6eece75185439d977d3a91942e8dfa; no voters:
I20260806 18:43:48.062960 4132 raft_consensus.cc:2781] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:48.063208 4132 raft_consensus.cc:686] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 1 LEADER]: Becoming Leader. State: Replica: fb6eece75185439d977d3a91942e8dfa, State: Running, Role: LEADER
I20260806 18:43:48.063573 4132 consensus_queue.cc:237] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } }
I20260806 18:43:48.066836 3394 catalog_manager.cc:5495] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa reported cstate change: term changed from 0 to 1, leader changed from <none> to fb6eece75185439d977d3a91942e8dfa (127.8.241.194). New cstate: current_term: 1 leader_uuid: "fb6eece75185439d977d3a91942e8dfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb6eece75185439d977d3a91942e8dfa" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 46665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 } health_report { overall_health: UNKNOWN } } }
W20260806 18:43:48.074415 3693 tablet.cc:2369] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:48.086755 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:48.083309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35806 (local address 127.8.241.194:46665)
0806 18:43:48.083451 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:48.083456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:48.083473 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:43:48.083550 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:48.083552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:48.083606 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:48.083693 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:43:48.083938 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.084439 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.085345 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:48.085566 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:48.085652 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:48.085958 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:43:48.086368 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:43:48.086472 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:48.086541 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:43:48.086596 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:43:48.092516 4025 raft_consensus.cc:1261] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [term 1 FOLLOWER]: Refusing update from remote peer fb6eece75185439d977d3a91942e8dfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:48.092545 3620 raft_consensus.cc:1261] T e8dc6f1b68b7473d8d89667508248d79 P bdd6430138eb4307b068c24acbb3bade [term 1 FOLLOWER]: Refusing update from remote peer fb6eece75185439d977d3a91942e8dfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:48.093245 4139 consensus_queue.cc:1035] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e5b6d295f7a400f8087e50764e40ddd" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 45129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:48.093423 4132 consensus_queue.cc:1035] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdd6430138eb4307b068c24acbb3bade" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:48.101148 4145 mvcc.cc:205] Tried to move back new op lower bound from 7315627327859462144 to 7315627327750795264. Current Snapshot: MvccSnapshot[applied={T|T < 7315627327859462144}]
I20260806 18:43:48.101284 4147 mvcc.cc:205] Tried to move back new op lower bound from 7315627327859462144 to 7315627327750795264. Current Snapshot: MvccSnapshot[applied={T|T < 7315627327859462144}]
I20260806 18:43:48.109242 3400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:48.109406 3400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:48.111783 3400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3be9954f41a94668b7980c256db57153: Sending DeleteTablet for 3 replicas of tablet e8dc6f1b68b7473d8d89667508248d79
I20260806 18:43:48.112609 3786 tablet_service.cc:1514] Processing DeleteTablet for tablet e8dc6f1b68b7473d8d89667508248d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:48 UTC) from {username='slave'} at 127.0.0.1:35796
I20260806 18:43:48.112617 3588 tablet_service.cc:1514] Processing DeleteTablet for tablet e8dc6f1b68b7473d8d89667508248d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:48 UTC) from {username='slave'} at 127.0.0.1:47966
I20260806 18:43:48.113161 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 3498
I20260806 18:43:48.113307 4001 tablet_service.cc:1514] Processing DeleteTablet for tablet e8dc6f1b68b7473d8d89667508248d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:48 UTC) from {username='slave'} at 127.0.0.1:52396
I20260806 18:43:48.119225 4168 tablet_replica.cc:330] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: stopping tablet replica
I20260806 18:43:48.120083 4168 raft_consensus.cc:2229] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:43:48.120576 4168 raft_consensus.cc:2258] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:48.124444 4171 tablet_replica.cc:330] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: stopping tablet replica
I20260806 18:43:48.124775 4171 raft_consensus.cc:2229] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:48.125137 4171 raft_consensus.cc:2258] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:48.125425 4168 ts_tablet_manager.cc:1905] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:48.128464 4168 ts_tablet_manager.cc:1918] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:48.128579 4168 log.cc:1196] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/e8dc6f1b68b7473d8d89667508248d79
I20260806 18:43:48.128927 4168 ts_tablet_manager.cc:1939] T e8dc6f1b68b7473d8d89667508248d79 P fb6eece75185439d977d3a91942e8dfa: Deleting consensus metadata
I20260806 18:43:48.130374 3376 catalog_manager.cc:4841] TS fb6eece75185439d977d3a91942e8dfa (127.8.241.194:46665): tablet e8dc6f1b68b7473d8d89667508248d79 (table test-table [id=c35ec0ef63294003b3f69eb26b99d056]) successfully deleted
I20260806 18:43:48.131284 4171 ts_tablet_manager.cc:1905] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:48.134004 4171 ts_tablet_manager.cc:1918] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260806 18:43:48.134068 3378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:33591 (error 108)
I20260806 18:43:48.134145 4171 log.cc:1196] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/e8dc6f1b68b7473d8d89667508248d79
I20260806 18:43:48.134289 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 3695
I20260806 18:43:48.134444 4171 ts_tablet_manager.cc:1939] T e8dc6f1b68b7473d8d89667508248d79 P 7e5b6d295f7a400f8087e50764e40ddd: Deleting consensus metadata
I20260806 18:43:48.136677 3378 catalog_manager.cc:4841] TS 7e5b6d295f7a400f8087e50764e40ddd (127.8.241.195:45129): tablet e8dc6f1b68b7473d8d89667508248d79 (table test-table [id=c35ec0ef63294003b3f69eb26b99d056]) successfully deleted
I20260806 18:43:48.139743 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:48.138484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:33591 (local address 127.0.0.1:47974)
0806 18:43:48.139565 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:48.139647 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:33591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":982}
W20260806 18:43:48.139937 3378 catalog_manager.cc:4584] TS bdd6430138eb4307b068c24acbb3bade (127.8.241.193:33591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e8dc6f1b68b7473d8d89667508248d79: Network error: Client connection negotiation failed: client connection to 127.8.241.193:33591: connect: Connection refused (error 111)
I20260806 18:43:48.145941 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 3900
I20260806 18:43:48.160187 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 3351
2026-08-06T18:43:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1066 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:43:48 dist-test-slave-k5rr krb5kdc[4177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:43:48 dist-test-slave-k5rr krb5kdc[4177](info): set up 2 sockets
Aug 06 18:43:48 dist-test-slave-k5rr krb5kdc[4177](info): commencing operation
krb5kdc: starting...
W20260806 18:43:49.285801 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:43:49 dist-test-slave-k5rr krb5kdc[4177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:43:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:43:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:43:49.399871 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:38621
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:38621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:49.511260 4199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:49.511564 4199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:49.511624 4199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:49.515650 4199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:49.515714 4199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:49.515740 4199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:49.515764 4199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:49.515792 4199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:49.521060 4199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:38621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:38621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:49.522118 4199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:49.523144 4199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:49.531414 4208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:49.531414 4206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:49.531483 4204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:49.531672 4199 server_base.cc:1034] running on GCE node
I20260806 18:43:49.532454 4199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:49.533574 4199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:49.534734 4199 hybrid_clock.cc:648] HybridClock initialized: now 1786041829534729 us; error 50 us; skew 500 ppm
Aug 06 18:43:49 dist-test-slave-k5rr krb5kdc[4177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041829, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:49.537303 4199 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:43:49.538148 4199 webserver.cc:460] Webserver started at http://127.8.241.254:33427/ using document root <none> and password file <none>
I20260806 18:43:49.538666 4199 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:49.538720 4199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:49.538897 4199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:49.540833 4199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "a647d5927e114a7cbb6fdc0c506cded2"
format_stamp: "Formatted at 2026-08-06 18:43:49 on dist-test-slave-k5rr"
server_key: "e9b3574422650a24c492fb49de6f387a"
server_key_iv: "24e75cf404c26e3b463d5abaf9e6a1b5"
server_key_version: "encryptionkey@0"
I20260806 18:43:49.541393 4199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "a647d5927e114a7cbb6fdc0c506cded2"
format_stamp: "Formatted at 2026-08-06 18:43:49 on dist-test-slave-k5rr"
server_key: "e9b3574422650a24c492fb49de6f387a"
server_key_iv: "24e75cf404c26e3b463d5abaf9e6a1b5"
server_key_version: "encryptionkey@0"
I20260806 18:43:49.544833 4199 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:43:49.547219 4221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:49.548440 4199 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:49.548561 4199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "a647d5927e114a7cbb6fdc0c506cded2"
format_stamp: "Formatted at 2026-08-06 18:43:49 on dist-test-slave-k5rr"
server_key: "e9b3574422650a24c492fb49de6f387a"
server_key_iv: "24e75cf404c26e3b463d5abaf9e6a1b5"
server_key_version: "encryptionkey@0"
I20260806 18:43:49.548658 4199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:49.565270 4199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:49.567919 4199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:49.568084 4199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:49.575042 4199 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:38621
I20260806 18:43:49.575071 4296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:38621 every 8 connection(s)
I20260806 18:43:49.575963 4199 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:49.578904 4298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:49.584465 4298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2: Bootstrap starting.
I20260806 18:43:49.585716 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 4199
I20260806 18:43:49.585852 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:43:49.586156 9159 external_mini_cluster.cc:1428] Setting key c3997d6e084f200eeeb8d163f4451250
I20260806 18:43:49.587350 4298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:49.588169 4298 log.cc:824] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:49.590102 4298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2: No bootstrap required, opened a new log
I20260806 18:43:49.593626 4298 raft_consensus.cc:348] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a647d5927e114a7cbb6fdc0c506cded2" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 38621 } }
I20260806 18:43:49.593883 4298 raft_consensus.cc:374] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:49.593951 4298 raft_consensus.cc:729] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a647d5927e114a7cbb6fdc0c506cded2, State: Initialized, Role: FOLLOWER
I20260806 18:43:49.594372 4298 consensus_queue.cc:260] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a647d5927e114a7cbb6fdc0c506cded2" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 38621 } }
I20260806 18:43:49.594506 4298 raft_consensus.cc:388] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 06 18:43:49 dist-test-slave-k5rr krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:49.594570 4298 raft_consensus.cc:482] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:49.594672 4298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:49.595551 4298 raft_consensus.cc:504] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a647d5927e114a7cbb6fdc0c506cded2" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 38621 } }
I20260806 18:43:49.595902 4298 leader_election.cc:302] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a647d5927e114a7cbb6fdc0c506cded2; no voters:
I20260806 18:43:49.596237 4298 leader_election.cc:288] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:43:49.596308 4304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:49.596508 4304 raft_consensus.cc:686] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [term 1 LEADER]: Becoming Leader. State: Replica: a647d5927e114a7cbb6fdc0c506cded2, State: Running, Role: LEADER
I20260806 18:43:49.596876 4304 consensus_queue.cc:237] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a647d5927e114a7cbb6fdc0c506cded2" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 38621 } }
I20260806 18:43:49.597641 4298 sys_catalog.cc:564] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:49.598634 4306 sys_catalog.cc:455] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a647d5927e114a7cbb6fdc0c506cded2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a647d5927e114a7cbb6fdc0c506cded2" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 38621 } } }
I20260806 18:43:49.598771 4306 sys_catalog.cc:458] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:49.599092 4307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a647d5927e114a7cbb6fdc0c506cded2. Latest consensus state: current_term: 1 leader_uuid: "a647d5927e114a7cbb6fdc0c506cded2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a647d5927e114a7cbb6fdc0c506cded2" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 38621 } } }
I20260806 18:43:49.599187 4307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:49.600170 4319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:49.600798 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:49.587331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52034 (local address 127.8.241.254:38621)
0806 18:43:49.587780 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:49.587796 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:49.587831 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:43:49.588580 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:49.588605 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:49.589040 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:49.589363 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:49.590157 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:49.591131 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:49.591972 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:49.592442 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:49.592589 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:49.595118 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:49.595123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:49.595136 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:49.595163 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:49.597100 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:49.597504 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:49.597510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:49.597516 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:49.597648 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:49.597910 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:49.597914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:49.597916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:49.598460 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:49.599377 (+ 917us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:49.599531 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 18:43:49.599839 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":146,"threads_started":1}
I20260806 18:43:49.603621 4319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:49.609838 4319 catalog_manager.cc:1347] Generated new cluster ID: 0b35153798bd466fb39e058e95808e36
I20260806 18:43:49.609905 4319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:49.615935 4319 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:43:49.616851 4319 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:49.626358 4319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2: Generated new TSK 0
I20260806 18:43:49.627060 4319 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:43:49.658177 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:38621
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:49.776296 4339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:49.776572 4339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:49.776629 4339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:49.780691 4339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:49.780774 4339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:49.780849 4339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:43:49.786226 4339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:38621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:49.787298 4339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:49.788362 4339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:49.798264 4357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:49.798223 4350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:49.798223 4349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:49.798390 4339 server_base.cc:1034] running on GCE node
I20260806 18:43:49.799064 4339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:49.799686 4339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:49.800855 4339 hybrid_clock.cc:648] HybridClock initialized: now 1786041829800830 us; error 35 us; skew 500 ppm
Aug 06 18:43:49 dist-test-slave-k5rr krb5kdc[4177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041829, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:49.803609 4339 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:43:49.804472 4339 webserver.cc:460] Webserver started at http://127.8.241.193:39105/ using document root <none> and password file <none>
I20260806 18:43:49.805078 4339 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:49.805152 4339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:49.805361 4339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:49.807400 4339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "07c1ae54388e4051b4d4b51e3789adea"
format_stamp: "Formatted at 2026-08-06 18:43:49 on dist-test-slave-k5rr"
server_key: "57b23b0324632b91e5d38e8682152ab7"
server_key_iv: "d18cbc518789baacea0e0f2b6af417ec"
server_key_version: "encryptionkey@0"
I20260806 18:43:49.807950 4339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "07c1ae54388e4051b4d4b51e3789adea"
format_stamp: "Formatted at 2026-08-06 18:43:49 on dist-test-slave-k5rr"
server_key: "57b23b0324632b91e5d38e8682152ab7"
server_key_iv: "d18cbc518789baacea0e0f2b6af417ec"
server_key_version: "encryptionkey@0"
I20260806 18:43:49.811607 4339 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:43:49.814085 4369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:49.815282 4339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:49.815392 4339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "07c1ae54388e4051b4d4b51e3789adea"
format_stamp: "Formatted at 2026-08-06 18:43:49 on dist-test-slave-k5rr"
server_key: "57b23b0324632b91e5d38e8682152ab7"
server_key_iv: "d18cbc518789baacea0e0f2b6af417ec"
server_key_version: "encryptionkey@0"
I20260806 18:43:49.815490 4339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:49.848285 4339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:49.851338 4339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:49.851576 4339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:49.852237 4339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:49.853192 4339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:49.853243 4339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:49.853297 4339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:49.853327 4339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:49.861980 4339 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:35745
I20260806 18:43:49.862017 4539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:35745 every 8 connection(s)
I20260806 18:43:49.862876 4339 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:43:49.865296 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 4339
I20260806 18:43:49.865433 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:43:49.865756 9159 external_mini_cluster.cc:1428] Setting key 7d9811290e4901bbcff9a4aca83f009d
Aug 06 18:43:49 dist-test-slave-k5rr krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041829, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:43:49.876571 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:49.864727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:57191 (local address 127.8.241.254:38621)
0806 18:43:49.864900 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:49.864905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:49.865855 (+ 950us) server_negotiation.cc:407] Connection header received
0806 18:43:49.867392 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:49.867396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:49.867451 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:49.867522 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:49.868709 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:49.869302 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:49.870436 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:49.870668 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:49.870757 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:49.873109 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:49.873113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:49.873115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:49.873140 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:49.874420 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:49.874968 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:49.874971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:49.874973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:49.875058 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:49.875430 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:49.875433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:49.875435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:49.875619 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:49.875707 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:49.876261 (+ 554us) server_negotiation.cc:299] Negotiation successful
0806 18:43:49.876390 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:43:49.877599 4542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:49.865182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:38621 (local address 127.8.241.193:57191)
0806 18:43:49.865719 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:49.865761 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:49.867159 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:49.867692 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:49.867700 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:49.868055 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:49.868554 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:49.868568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:49.869428 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:49.869431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:49.870322 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:49.870332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:49.870807 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:49.870811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:49.870973 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:49.871514 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:49.871530 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:49.872926 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:49.874529 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:49.874536 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:49.874551 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:49.874875 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:49.875145 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:49.875148 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:49.875150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:49.875338 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:49.875721 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:49.875728 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:49.876026 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 18:43:49.876200 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 18:43:49.876564 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":152,"threads_started":1}
I20260806 18:43:49.878970 4540 heartbeater.cc:347] Connected to a master server at 127.8.241.254:38621
I20260806 18:43:49.879247 4540 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:49.879843 4540 heartbeater.cc:511] Master 127.8.241.254:38621 requested a full tablet report, sending...
I20260806 18:43:49.881544 4244 ts_manager.cc:194] Registered new tserver with Master: 07c1ae54388e4051b4d4b51e3789adea (127.8.241.193:35745)
I20260806 18:43:49.882709 4244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:57191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:43:49.893051 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:38621
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:50.006585 4548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:50.006826 4548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:50.006866 4548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:50.010787 4548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:50.010833 4548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:50.010887 4548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:43:50.015937 4548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:38621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:50.016896 4548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:50.017897 4548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:50.027297 4559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:50.027309 4556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:50.027314 4555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:50.027482 4548 server_base.cc:1034] running on GCE node
I20260806 18:43:50.028002 4548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:50.028636 4548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:50.029790 4548 hybrid_clock.cc:648] HybridClock initialized: now 1786041830029772 us; error 29 us; skew 500 ppm
Aug 06 18:43:50 dist-test-slave-k5rr krb5kdc[4177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041830, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:50.032356 4548 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:43:50.033296 4548 webserver.cc:460] Webserver started at http://127.8.241.194:43419/ using document root <none> and password file <none>
I20260806 18:43:50.033829 4548 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:50.033895 4548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:50.034081 4548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:50.036103 4548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "338a7f73204a7061a75ac3d4b4effd57"
server_key_iv: "5ed865f29a83be3f2dbd46abaa4ef9b1"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.036654 4548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "338a7f73204a7061a75ac3d4b4effd57"
server_key_iv: "5ed865f29a83be3f2dbd46abaa4ef9b1"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.040318 4548 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:43:50.042876 4571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.044064 4548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:50.044176 4548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "338a7f73204a7061a75ac3d4b4effd57"
server_key_iv: "5ed865f29a83be3f2dbd46abaa4ef9b1"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.044274 4548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:50.055878 4548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:50.058601 4548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:50.058782 4548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:50.059370 4548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:50.060209 4548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:50.060268 4548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.060324 4548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:50.060351 4548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.069165 4548 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:39401
I20260806 18:43:50.069192 4744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:39401 every 8 connection(s)
I20260806 18:43:50.070036 4548 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
Aug 06 18:43:50 dist-test-slave-k5rr krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041830, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:50.079905 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 4548
I20260806 18:43:50.080027 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:43:50.080358 9159 external_mini_cluster.cc:1428] Setting key 19a055590a605a4b8d70e9fe9ec5d77d
I20260806 18:43:50.083586 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.071930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:46231 (local address 127.8.241.254:38621)
0806 18:43:50.072074 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.072078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.072931 (+ 853us) server_negotiation.cc:407] Connection header received
0806 18:43:50.074302 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.074305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.074363 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.074444 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:50.075664 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.076196 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.077353 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.077576 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.077666 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.080105 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:50.080110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:50.080113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:50.080145 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:50.081593 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:50.082091 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:50.082094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:50.082096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:50.082173 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:50.082493 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:50.082497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:50.082499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:50.082679 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:50.082763 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.083279 (+ 516us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.083414 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:43:50.084427 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.072318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:38621 (local address 127.8.241.194:46231)
0806 18:43:50.072772 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:50.072814 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:50.074087 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:50.074629 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:50.074635 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:50.074983 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:50.075524 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.075541 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.076318 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.076320 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.077233 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.077243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.077720 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.077724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.077888 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.078413 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:50.078429 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:50.079917 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:50.081709 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:50.081713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:50.081726 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:50.081992 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:50.082257 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:50.082260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:50.082262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:50.082403 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:50.082787 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:50.082794 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:50.083038 (+ 244us) client_negotiation.cc:769] Sending connection context
0806 18:43:50.083206 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 18:43:50.083534 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":106,"threads_started":1}
I20260806 18:43:50.085652 4746 heartbeater.cc:347] Connected to a master server at 127.8.241.254:38621
I20260806 18:43:50.085901 4746 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:50.086401 4746 heartbeater.cc:511] Master 127.8.241.254:38621 requested a full tablet report, sending...
I20260806 18:43:50.087558 4244 ts_manager.cc:194] Registered new tserver with Master: c9aa71d8ea1943588ea7d56bb1bf85d3 (127.8.241.194:39401)
I20260806 18:43:50.088189 4244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:46231
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:43:50.106254 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:38621
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:40861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:50.218719 4753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:50.218972 4753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:50.219020 4753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:50.223063 4753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:50.223116 4753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:50.223177 4753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:43:50.228559 4753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:40861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:38621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:50.229633 4753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:50.230672 4753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:50.240295 4768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:50.240304 4761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:50.240296 4764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:50.240434 4753 server_base.cc:1034] running on GCE node
I20260806 18:43:50.241114 4753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:50.241765 4753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:50.242950 4753 hybrid_clock.cc:648] HybridClock initialized: now 1786041830242918 us; error 43 us; skew 500 ppm
Aug 06 18:43:50 dist-test-slave-k5rr krb5kdc[4177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041830, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:50.245648 4753 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:43:50.246505 4753 webserver.cc:460] Webserver started at http://127.8.241.195:37201/ using document root <none> and password file <none>
I20260806 18:43:50.247072 4753 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:50.247141 4753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:50.247339 4753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:50.249413 4753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "ae2a6f751d0f42789bc4b0b124135b3f"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "45ee0b07e45060fa33b0fa68b2db3c13"
server_key_iv: "bc79dfaabb00343666e74868dbe12ca9"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.249949 4753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "ae2a6f751d0f42789bc4b0b124135b3f"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "45ee0b07e45060fa33b0fa68b2db3c13"
server_key_iv: "bc79dfaabb00343666e74868dbe12ca9"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.253779 4753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:43:50.256467 4780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.257670 4753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:50.257778 4753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "ae2a6f751d0f42789bc4b0b124135b3f"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "45ee0b07e45060fa33b0fa68b2db3c13"
server_key_iv: "bc79dfaabb00343666e74868dbe12ca9"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.257881 4753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:50.271636 4753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:50.274371 4753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:50.274541 4753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:50.275112 4753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:50.275980 4753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:50.276029 4753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.276083 4753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:50.276113 4753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.284967 4753 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:37067
I20260806 18:43:50.284998 4954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:37067 every 8 connection(s)
I20260806 18:43:50.285897 4753 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:43:50.292691 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 4753
I20260806 18:43:50.292824 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:43:50.293159 9159 external_mini_cluster.cc:1428] Setting key 6fc4212dce7a4ad0199ad04298f11639
Aug 06 18:43:50 dist-test-slave-k5rr krb5kdc[4177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041830, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:50.300453 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.287654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:58949 (local address 127.8.241.254:38621)
0806 18:43:50.287793 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.287798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.288623 (+ 825us) server_negotiation.cc:407] Connection header received
0806 18:43:50.290039 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.290042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.290102 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.290174 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:50.291362 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.291908 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.293369 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.293640 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.293746 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.296585 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:50.296590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:50.296592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:50.296623 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:50.298693 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:50.299191 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:50.299194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:50.299195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:50.299269 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:50.299537 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:50.299539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:50.299540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:50.299701 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:50.299771 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.300226 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.300322 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:43:50.301364 4958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.288015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:38621 (local address 127.8.241.195:58949)
0806 18:43:50.288480 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:50.288523 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:50.289820 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:50.290358 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:50.290365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:50.290719 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:50.291217 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.291230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.292033 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.292036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.293240 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.293251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.293801 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.293806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.293992 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.294596 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:50.294618 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:50.296400 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:50.298780 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:50.298784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:50.298798 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:50.299115 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:50.299332 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:50.299334 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:50.299336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:50.299472 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:50.299772 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:50.299777 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:50.300008 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 18:43:50.300177 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 18:43:50.300444 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":116,"threads_started":1}
I20260806 18:43:50.302413 4955 heartbeater.cc:347] Connected to a master server at 127.8.241.254:38621
I20260806 18:43:50.302641 4955 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:50.303123 4955 heartbeater.cc:511] Master 127.8.241.254:38621 requested a full tablet report, sending...
I20260806 18:43:50.304097 4244 ts_manager.cc:194] Registered new tserver with Master: ae2a6f751d0f42789bc4b0b124135b3f (127.8.241.195:37067)
I20260806 18:43:50.304621 4244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:58949
I20260806 18:43:50.307021 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:43:50.315984 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.308915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52042 (local address 127.8.241.254:38621)
0806 18:43:50.309058 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.309062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.309224 (+ 162us) server_negotiation.cc:407] Connection header received
0806 18:43:50.309339 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.309342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.309384 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.309446 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:50.310116 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.310607 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.311384 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.311613 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.311695 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.313131 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:50.313135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:50.313137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:50.313163 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:50.314328 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:50.314789 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:50.314792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:50.314794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:50.314870 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:50.315174 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:50.315177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:50.315178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:50.315366 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:50.315502 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.315755 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.315839 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:43:50.318635 4244 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:43:50.321753 4244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:43:50.335773 4976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.330120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:35745 (local address 127.0.0.1:35478)
0806 18:43:50.330355 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:50.330368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:50.330460 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:50.331422 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:50.331425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:50.331443 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:50.331577 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.331585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.332937 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.332940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.334229 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.334239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.335264 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.335268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.335396 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.335435 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:43:50.335543 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 18:43:50.335652 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":41,"thread_start_us":81,"threads_started":1}
I20260806 18:43:50.335940 4974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.329810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:39401 (local address 127.0.0.1:53762)
0806 18:43:50.330130 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:50.330176 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:50.330296 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:50.331352 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:50.331359 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:50.331379 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:50.331705 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.331710 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.333113 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.333116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.334439 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.334453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.335515 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.335518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.335607 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.335630 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:43:50.335709 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:43:50.335777 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":57,"thread_start_us":84,"threads_started":1}
I20260806 18:43:50.335981 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.329960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.8.241.194:39401)
0806 18:43:50.330818 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.330829 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.330854 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:43:50.330935 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.330941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.331144 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.331343 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:50.331816 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.332896 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.334585 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.335364 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.335501 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.335641 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.335740 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.335838 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":107,"threads_started":1}
I20260806 18:43:50.336287 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.330531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37067 (local address 127.0.0.1:55712)
0806 18:43:50.331274 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:50.331287 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:50.331377 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:50.332015 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:50.332017 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:50.332034 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:50.332111 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.332117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.333133 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.333137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.334473 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.334481 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.335269 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.335273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.335987 (+ 714us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.336013 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:43:50.336084 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:43:50.336150 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":675,"thread_start_us":95,"threads_started":1}
I20260806 18:43:50.336764 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.330631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.8.241.193:35745)
0806 18:43:50.331001 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.331016 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.331036 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:43:50.331100 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.331104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.331277 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.331536 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:50.331669 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.332803 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.334343 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.335120 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.335232 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.335399 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.336537 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.336624 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":75,"threads_started":1}
I20260806 18:43:50.339228 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.330874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.8.241.195:37067)
0806 18:43:50.331628 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.331640 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.331662 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:43:50.331731 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.331736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.331901 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.332023 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:50.332218 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.332949 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.334592 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.335147 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.335383 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.335527 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.336506 (+ 979us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.336608 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":151,"threads_started":1}
I20260806 18:43:50.338881 4644 tablet_service.cc:1467] Processing CreateTablet for tablet 8a18d2cc6d254864b8e37b4bc3ebc244 (DEFAULT_TABLE table=test-table [id=0312984dcf92448f99706eaf6eb17afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:50.339157 4440 tablet_service.cc:1467] Processing CreateTablet for tablet 8a18d2cc6d254864b8e37b4bc3ebc244 (DEFAULT_TABLE table=test-table [id=0312984dcf92448f99706eaf6eb17afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:50.340107 4644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a18d2cc6d254864b8e37b4bc3ebc244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:50.340113 4440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a18d2cc6d254864b8e37b4bc3ebc244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:50.342136 4850 tablet_service.cc:1467] Processing CreateTablet for tablet 8a18d2cc6d254864b8e37b4bc3ebc244 (DEFAULT_TABLE table=test-table [id=0312984dcf92448f99706eaf6eb17afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:50.343166 4850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a18d2cc6d254864b8e37b4bc3ebc244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:50.346879 4982 tablet_bootstrap.cc:492] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: Bootstrap starting.
I20260806 18:43:50.348093 4984 tablet_bootstrap.cc:492] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea: Bootstrap starting.
I20260806 18:43:50.349520 4982 tablet_bootstrap.cc:654] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:50.350278 4982 log.cc:824] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:50.350313 4984 tablet_bootstrap.cc:654] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:50.350427 4986 tablet_bootstrap.cc:492] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: Bootstrap starting.
I20260806 18:43:50.351663 4984 log.cc:824] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:50.352401 4982 tablet_bootstrap.cc:492] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: No bootstrap required, opened a new log
I20260806 18:43:50.352547 4986 tablet_bootstrap.cc:654] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:50.352582 4982 ts_tablet_manager.cc:1397] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:43:50.353708 4986 log.cc:824] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:50.354576 4984 tablet_bootstrap.cc:492] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea: No bootstrap required, opened a new log
I20260806 18:43:50.354760 4984 ts_tablet_manager.cc:1397] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260806 18:43:50.355597 4986 tablet_bootstrap.cc:492] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: No bootstrap required, opened a new log
I20260806 18:43:50.355770 4986 ts_tablet_manager.cc:1397] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:43:50.355751 4982 raft_consensus.cc:348] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } }
I20260806 18:43:50.356014 4982 raft_consensus.cc:374] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:50.356070 4982 raft_consensus.cc:729] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9aa71d8ea1943588ea7d56bb1bf85d3, State: Initialized, Role: FOLLOWER
I20260806 18:43:50.356485 4982 consensus_queue.cc:260] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } }
I20260806 18:43:50.357168 4746 heartbeater.cc:503] Master 127.8.241.254:38621 was elected leader, sending a full tablet report...
I20260806 18:43:50.357434 4982 ts_tablet_manager.cc:1428] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:43:50.358242 4984 raft_consensus.cc:348] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } }
I20260806 18:43:50.358480 4984 raft_consensus.cc:374] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:50.358515 4984 raft_consensus.cc:729] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c1ae54388e4051b4d4b51e3789adea, State: Initialized, Role: FOLLOWER
I20260806 18:43:50.358917 4984 consensus_queue.cc:260] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } }
I20260806 18:43:50.359505 4540 heartbeater.cc:503] Master 127.8.241.254:38621 was elected leader, sending a full tablet report...
I20260806 18:43:50.359784 4984 ts_tablet_manager.cc:1428] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:43:50.359663 4986 raft_consensus.cc:348] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } }
I20260806 18:43:50.359870 4986 raft_consensus.cc:374] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:50.359928 4986 raft_consensus.cc:729] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2a6f751d0f42789bc4b0b124135b3f, State: Initialized, Role: FOLLOWER
I20260806 18:43:50.360311 4986 consensus_queue.cc:260] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } }
I20260806 18:43:50.361063 4955 heartbeater.cc:503] Master 127.8.241.254:38621 was elected leader, sending a full tablet report...
I20260806 18:43:50.361240 4986 ts_tablet_manager.cc:1428] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260806 18:43:50.364023 4541 tablet.cc:2369] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:50.382000 4994 raft_consensus.cc:482] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:50.382195 4994 raft_consensus.cc:504] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } }
I20260806 18:43:50.383397 4994 leader_election.cc:288] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9aa71d8ea1943588ea7d56bb1bf85d3 (127.8.241.194:39401), 07c1ae54388e4051b4d4b51e3789adea (127.8.241.193:35745)
I20260806 18:43:50.387223 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.383720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:36601 (local address 127.8.241.194:39401)
0806 18:43:50.383896 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.383901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.383918 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:43:50.383977 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.383980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.384027 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.384114 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:50.384496 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.385120 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.386226 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.386658 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.386750 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.386793 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.387025 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.387101 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260806 18:43:50.387228 4958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.383616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:39401 (local address 127.8.241.195:36601)
0806 18:43:50.383793 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:50.383808 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:50.383898 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:50.384167 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:50.384171 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:50.384198 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:50.384348 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.384356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.385240 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.385244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.386089 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.386100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.386774 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.386778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.386889 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.386919 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 18:43:50.387001 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:43:50.387063 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260806 18:43:50.387260 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.383644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:35745 (local address 127.8.241.195:47535)
0806 18:43:50.384079 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:50.384093 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:50.384186 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:50.384470 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:50.384473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:50.384494 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:50.384608 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.384614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.385453 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.385456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.386229 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.386238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.386897 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.386900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.386983 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.387008 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:43:50.387082 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:43:50.387143 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":146,"threads_started":1}
I20260806 18:43:50.387302 4980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.383793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:47535 (local address 127.8.241.193:35745)
0806 18:43:50.383910 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.383914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.384196 (+ 282us) server_negotiation.cc:407] Connection header received
0806 18:43:50.384270 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.384273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.384322 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.384427 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:50.384753 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.385336 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.386351 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.386773 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.386901 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.386952 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.387110 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.387185 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:43:50.387894 4469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a18d2cc6d254864b8e37b4bc3ebc244" candidate_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c1ae54388e4051b4d4b51e3789adea" is_pre_election: true
I20260806 18:43:50.387902 4675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a18d2cc6d254864b8e37b4bc3ebc244" candidate_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" is_pre_election: true
I20260806 18:43:50.388190 4469 raft_consensus.cc:2454] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae2a6f751d0f42789bc4b0b124135b3f in term 0.
I20260806 18:43:50.388190 4675 raft_consensus.cc:2454] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae2a6f751d0f42789bc4b0b124135b3f in term 0.
I20260806 18:43:50.388643 4781 leader_election.cc:302] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07c1ae54388e4051b4d4b51e3789adea, ae2a6f751d0f42789bc4b0b124135b3f; no voters:
I20260806 18:43:50.388893 4994 raft_consensus.cc:2781] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:43:50.388983 4994 raft_consensus.cc:482] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:43:50.389055 4994 raft_consensus.cc:3037] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:50.390050 4994 raft_consensus.cc:504] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } }
I20260806 18:43:50.390395 4994 leader_election.cc:288] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [CANDIDATE]: Term 1 election: Requested vote from peers c9aa71d8ea1943588ea7d56bb1bf85d3 (127.8.241.194:39401), 07c1ae54388e4051b4d4b51e3789adea (127.8.241.193:35745)
I20260806 18:43:50.390724 4469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a18d2cc6d254864b8e37b4bc3ebc244" candidate_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c1ae54388e4051b4d4b51e3789adea"
I20260806 18:43:50.390724 4675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a18d2cc6d254864b8e37b4bc3ebc244" candidate_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3"
I20260806 18:43:50.390839 4469 raft_consensus.cc:3037] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:50.390847 4675 raft_consensus.cc:3037] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:50.391888 4469 raft_consensus.cc:2454] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae2a6f751d0f42789bc4b0b124135b3f in term 1.
I20260806 18:43:50.392026 4675 raft_consensus.cc:2454] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae2a6f751d0f42789bc4b0b124135b3f in term 1.
I20260806 18:43:50.392174 4781 leader_election.cc:302] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07c1ae54388e4051b4d4b51e3789adea, ae2a6f751d0f42789bc4b0b124135b3f; no voters:
I20260806 18:43:50.392340 4994 raft_consensus.cc:2781] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:50.392544 4994 raft_consensus.cc:686] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 1 LEADER]: Becoming Leader. State: Replica: ae2a6f751d0f42789bc4b0b124135b3f, State: Running, Role: LEADER
I20260806 18:43:50.392874 4994 consensus_queue.cc:237] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } }
I20260806 18:43:50.395898 4244 catalog_manager.cc:5495] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f reported cstate change: term changed from 0 to 1, leader changed from <none> to ae2a6f751d0f42789bc4b0b124135b3f (127.8.241.195). New cstate: current_term: 1 leader_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae2a6f751d0f42789bc4b0b124135b3f" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260806 18:43:50.415096 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.411359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.8.241.195:37067)
0806 18:43:50.411497 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.411502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.411518 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:43:50.411632 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.411635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.411692 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.411805 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:43:50.412084 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.412597 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.413530 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.413748 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.413846 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.414184 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:43:50.414614 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:43:50.414762 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.414906 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.414947 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:43:50.420899 4469 raft_consensus.cc:1261] T 8a18d2cc6d254864b8e37b4bc3ebc244 P 07c1ae54388e4051b4d4b51e3789adea [term 1 FOLLOWER]: Refusing update from remote peer ae2a6f751d0f42789bc4b0b124135b3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:50.420895 4675 raft_consensus.cc:1261] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [term 1 FOLLOWER]: Refusing update from remote peer ae2a6f751d0f42789bc4b0b124135b3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:50.421592 4994 consensus_queue.cc:1035] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9aa71d8ea1943588ea7d56bb1bf85d3" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 39401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:50.421793 5001 consensus_queue.cc:1035] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "07c1ae54388e4051b4d4b51e3789adea" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:50.429672 5005 mvcc.cc:205] Tried to move back new op lower bound from 7315627337397207040 to 7315627337291870208. Current Snapshot: MvccSnapshot[applied={T|T < 7315627337397207040}]
I20260806 18:43:50.440250 4241 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:50.440404 4241 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:50.442754 4241 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a647d5927e114a7cbb6fdc0c506cded2: Sending DeleteTablet for 3 replicas of tablet 8a18d2cc6d254864b8e37b4bc3ebc244
I20260806 18:43:50.443593 4850 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a18d2cc6d254864b8e37b4bc3ebc244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:50 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:55712
I20260806 18:43:50.443585 4440 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a18d2cc6d254864b8e37b4bc3ebc244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:50 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:35478
I20260806 18:43:50.443652 4644 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a18d2cc6d254864b8e37b4bc3ebc244 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:50 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:53762
I20260806 18:43:50.444095 5016 tablet_replica.cc:330] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: stopping tablet replica
I20260806 18:43:50.444146 5017 tablet_replica.cc:330] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: stopping tablet replica
I20260806 18:43:50.444267 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 4339
I20260806 18:43:50.444404 5016 raft_consensus.cc:2229] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:43:50.444737 5016 raft_consensus.cc:2258] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:50.446630 5017 raft_consensus.cc:2229] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:50.446990 5017 raft_consensus.cc:2258] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:50.454324 5016 ts_tablet_manager.cc:1905] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:50.454770 5017 ts_tablet_manager.cc:1905] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:50.457115 5016 ts_tablet_manager.cc:1918] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:50.457216 5016 log.cc:1196] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/8a18d2cc6d254864b8e37b4bc3ebc244
I20260806 18:43:50.457513 5016 ts_tablet_manager.cc:1939] T 8a18d2cc6d254864b8e37b4bc3ebc244 P ae2a6f751d0f42789bc4b0b124135b3f: Deleting consensus metadata
I20260806 18:43:50.457787 5017 ts_tablet_manager.cc:1918] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:50.457950 5017 log.cc:1196] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/8a18d2cc6d254864b8e37b4bc3ebc244
I20260806 18:43:50.458344 5017 ts_tablet_manager.cc:1939] T 8a18d2cc6d254864b8e37b4bc3ebc244 P c9aa71d8ea1943588ea7d56bb1bf85d3: Deleting consensus metadata
I20260806 18:43:50.458761 4227 catalog_manager.cc:4841] TS ae2a6f751d0f42789bc4b0b124135b3f (127.8.241.195:37067): tablet 8a18d2cc6d254864b8e37b4bc3ebc244 (table test-table [id=0312984dcf92448f99706eaf6eb17afc]) successfully deleted
I20260806 18:43:50.459640 4227 catalog_manager.cc:4841] TS c9aa71d8ea1943588ea7d56bb1bf85d3 (127.8.241.194:39401): tablet 8a18d2cc6d254864b8e37b4bc3ebc244 (table test-table [id=0312984dcf92448f99706eaf6eb17afc]) successfully deleted
I20260806 18:43:50.460932 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 4548
W20260806 18:43:50.460903 4223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:35745 (error 108)
I20260806 18:43:50.464244 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.463945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:35745 (local address 127.0.0.1:35488)
0806 18:43:50.464082 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:50.464156 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:35745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 18:43:50.464423 4223 catalog_manager.cc:4584] TS 07c1ae54388e4051b4d4b51e3789adea (127.8.241.193:35745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a18d2cc6d254864b8e37b4bc3ebc244: Network error: Client connection negotiation failed: client connection to 127.8.241.193:35745: connect: Connection refused (error 111)
I20260806 18:43:50.469163 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 4753
I20260806 18:43:50.478029 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 4199
2026-08-06T18:43:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2315 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T18:43:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:43:50Z Disabled control of system clock
I20260806 18:43:50.513659 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37623
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:37623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:43:50.626657 5031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:50.626953 5031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:50.626991 5031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:50.630927 5031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:50.630995 5031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:50.631013 5031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:50.631034 5031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:50.631058 5031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:50.635885 5031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:37623
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:37623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:50.637081 5031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:50.638540 5031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:50.647011 5042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:50.647004 5040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:50.647004 5039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:50.647218 5031 server_base.cc:1034] running on GCE node
I20260806 18:43:50.647826 5031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:50.648945 5031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:50.650110 5031 hybrid_clock.cc:648] HybridClock initialized: now 1786041830650082 us; error 37 us; skew 500 ppm
I20260806 18:43:50.651901 5031 webserver.cc:460] Webserver started at http://127.8.241.254:42047/ using document root <none> and password file <none>
I20260806 18:43:50.652460 5031 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:50.652521 5031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:50.652726 5031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:50.654897 5031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "1dae28d5a23b43fa8aef8453a95ced5d"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "eb7aaf325b2a102562b8a5b0711a8422"
server_key_iv: "da7348e07b6fe87970ff7eb0149b3654"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.655444 5031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "1dae28d5a23b43fa8aef8453a95ced5d"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "eb7aaf325b2a102562b8a5b0711a8422"
server_key_iv: "da7348e07b6fe87970ff7eb0149b3654"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.659111 5031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:43:50.661429 5051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.662602 5031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:50.662706 5031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "1dae28d5a23b43fa8aef8453a95ced5d"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "eb7aaf325b2a102562b8a5b0711a8422"
server_key_iv: "da7348e07b6fe87970ff7eb0149b3654"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.662810 5031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:50.685881 5031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:50.686712 5031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:50.686929 5031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:50.694469 5031 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:37623
I20260806 18:43:50.694486 5123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:37623 every 8 connection(s)
I20260806 18:43:50.695530 5031 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:50.698769 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 5031
I20260806 18:43:50.698872 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:43:50.698884 5124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:50.699193 9159 external_mini_cluster.cc:1428] Setting key c150851871003a0f48928f9a5b30ae08
I20260806 18:43:50.705348 5124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d: Bootstrap starting.
I20260806 18:43:50.707665 5124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:50.708456 5124 log.cc:824] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:50.708747 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.700714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.8.241.254:37623)
0806 18:43:50.701444 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.701460 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.701498 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:43:50.702220 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.702241 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.702637 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.702968 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:50.703451 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.704436 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.705376 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.705913 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.706061 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.706969 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:50.706974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:50.706987 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:50.707380 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:50.707437 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:50.707458 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:50.707542 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.707682 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.707969 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":143,"threads_started":1}
I20260806 18:43:50.710544 5124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d: No bootstrap required, opened a new log
I20260806 18:43:50.714249 5124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dae28d5a23b43fa8aef8453a95ced5d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37623 } }
I20260806 18:43:50.714488 5124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:50.714546 5124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dae28d5a23b43fa8aef8453a95ced5d, State: Initialized, Role: FOLLOWER
I20260806 18:43:50.714978 5124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dae28d5a23b43fa8aef8453a95ced5d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37623 } }
I20260806 18:43:50.715118 5124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:50.715185 5124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:50.715286 5124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:50.716157 5124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dae28d5a23b43fa8aef8453a95ced5d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37623 } }
I20260806 18:43:50.716491 5124 leader_election.cc:302] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dae28d5a23b43fa8aef8453a95ced5d; no voters:
I20260806 18:43:50.716813 5124 leader_election.cc:288] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:43:50.716933 5131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:50.717236 5131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [term 1 LEADER]: Becoming Leader. State: Replica: 1dae28d5a23b43fa8aef8453a95ced5d, State: Running, Role: LEADER
I20260806 18:43:50.717540 5131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dae28d5a23b43fa8aef8453a95ced5d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37623 } }
I20260806 18:43:50.718055 5124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:50.719269 5133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dae28d5a23b43fa8aef8453a95ced5d. Latest consensus state: current_term: 1 leader_uuid: "1dae28d5a23b43fa8aef8453a95ced5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dae28d5a23b43fa8aef8453a95ced5d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37623 } } }
I20260806 18:43:50.719304 5132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dae28d5a23b43fa8aef8453a95ced5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dae28d5a23b43fa8aef8453a95ced5d" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 37623 } } }
I20260806 18:43:50.719408 5133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:50.719412 5132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:50.719870 5141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:50.722913 5141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:50.728207 5141 catalog_manager.cc:1347] Generated new cluster ID: 0a3a7a2eb5b84e71b3635b89d3bc78ba
I20260806 18:43:50.728271 5141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:50.737244 5141 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:43:50.738262 5141 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:50.743362 5141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d: Generated new TSK 0
I20260806 18:43:50.744028 5141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:43:50.746963 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37623
--builtin_ntp_servers=127.8.241.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:43:50.863193 5151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:50.863433 5151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:50.863483 5151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:50.867468 5151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:50.867517 5151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:50.867583 5151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:43:50.872263 5151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:50.873340 5151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:50.874682 5151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:43:50.884349 5151 server_base.cc:1034] running on GCE node
W20260806 18:43:50.884306 5161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:50.884306 5157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:50.884310 5156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:50.884943 5151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:50.885622 5151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:50.886782 5151 hybrid_clock.cc:648] HybridClock initialized: now 1786041830886763 us; error 34 us; skew 500 ppm
I20260806 18:43:50.888583 5151 webserver.cc:460] Webserver started at http://127.8.241.193:40263/ using document root <none> and password file <none>
I20260806 18:43:50.889199 5151 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:50.889268 5151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:50.889485 5151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:50.891530 5151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "0fba2ba55f2a45fd9fcc400a2869644e"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "e703dfb5e145793fc01601635d12d8f3"
server_key_iv: "5c13cf562ff6c3c0e07ce00a0af1950b"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.892055 5151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "0fba2ba55f2a45fd9fcc400a2869644e"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "e703dfb5e145793fc01601635d12d8f3"
server_key_iv: "5c13cf562ff6c3c0e07ce00a0af1950b"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.895620 5151 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260806 18:43:50.897976 5170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.899111 5151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:50.899214 5151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "0fba2ba55f2a45fd9fcc400a2869644e"
format_stamp: "Formatted at 2026-08-06 18:43:50 on dist-test-slave-k5rr"
server_key: "e703dfb5e145793fc01601635d12d8f3"
server_key_iv: "5c13cf562ff6c3c0e07ce00a0af1950b"
server_key_version: "encryptionkey@0"
I20260806 18:43:50.899303 5151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:50.913949 5151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:50.914610 5151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:50.914791 5151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:50.915377 5151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:50.916325 5151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:50.916383 5151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.916437 5151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:50.916466 5151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:50.925139 5151 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:38361
I20260806 18:43:50.925199 5344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:38361 every 8 connection(s)
I20260806 18:43:50.926031 5151 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:43:50.933859 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 5151
I20260806 18:43:50.934000 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:43:50.934338 9159 external_mini_cluster.cc:1428] Setting key cd29f59fcb6f5315ea3c2b497738f2d9
I20260806 18:43:50.935091 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.927863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:35919 (local address 127.8.241.254:37623)
0806 18:43:50.928023 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:50.928028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:50.929114 (+ 1086us) server_negotiation.cc:407] Connection header received
0806 18:43:50.930468 (+ 1354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:50.930470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:50.930517 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:50.930589 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:50.931480 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.932022 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.933148 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.933365 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.933439 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.934618 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:50.934621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:50.934623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:50.934635 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:50.934655 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:50.934663 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:50.934718 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:50.934907 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 18:43:50.934976 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:43:50.936065 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:50.928260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37623 (local address 127.8.241.193:35919)
0806 18:43:50.928954 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:50.928995 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:50.930289 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:50.930779 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:50.930787 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:50.930870 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:50.931360 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.931373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.932136 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.932139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.933054 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:50.933064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:50.933481 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:50.933483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:50.933645 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:50.934322 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:43:50.934324 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:50.934380 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:43:50.934464 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:50.934749 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:50.934758 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:50.934765 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:43:50.934867 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 18:43:50.935148 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":107,"threads_started":1}
I20260806 18:43:50.936458 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37623
--builtin_ntp_servers=127.8.241.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:43:50.937289 5345 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37623
I20260806 18:43:50.937515 5345 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:50.938074 5345 heartbeater.cc:511] Master 127.8.241.254:37623 requested a full tablet report, sending...
I20260806 18:43:50.939637 5072 ts_manager.cc:194] Registered new tserver with Master: 0fba2ba55f2a45fd9fcc400a2869644e (127.8.241.193:38361)
I20260806 18:43:50.940783 5072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.193:35919
W20260806 18:43:51.051028 5350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:51.051285 5350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:51.051322 5350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:51.055356 5350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:51.055413 5350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:51.055467 5350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:43:51.060210 5350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:51.061369 5350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:51.062778 5350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:51.072283 5359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:51.072326 5361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:51.072309 5363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:51.072631 5350 server_base.cc:1034] running on GCE node
I20260806 18:43:51.073108 5350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:51.073719 5350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:51.074875 5350 hybrid_clock.cc:648] HybridClock initialized: now 1786041831074849 us; error 40 us; skew 500 ppm
I20260806 18:43:51.076653 5350 webserver.cc:460] Webserver started at http://127.8.241.194:38003/ using document root <none> and password file <none>
I20260806 18:43:51.077296 5350 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:51.077364 5350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:51.077562 5350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:51.079597 5350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "2176fa188ab849fd9fd74bdb3e5f0f69"
format_stamp: "Formatted at 2026-08-06 18:43:51 on dist-test-slave-k5rr"
server_key: "c7237cab0e50a66d1cca2961c8cc805f"
server_key_iv: "543d5466225e150a9cafd2e500174fb4"
server_key_version: "encryptionkey@0"
I20260806 18:43:51.080147 5350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "2176fa188ab849fd9fd74bdb3e5f0f69"
format_stamp: "Formatted at 2026-08-06 18:43:51 on dist-test-slave-k5rr"
server_key: "c7237cab0e50a66d1cca2961c8cc805f"
server_key_iv: "543d5466225e150a9cafd2e500174fb4"
server_key_version: "encryptionkey@0"
I20260806 18:43:51.083969 5350 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:43:51.086436 5371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:51.087630 5350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:43:51.087734 5350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "2176fa188ab849fd9fd74bdb3e5f0f69"
format_stamp: "Formatted at 2026-08-06 18:43:51 on dist-test-slave-k5rr"
server_key: "c7237cab0e50a66d1cca2961c8cc805f"
server_key_iv: "543d5466225e150a9cafd2e500174fb4"
server_key_version: "encryptionkey@0"
I20260806 18:43:51.087831 5350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:51.099915 5350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:51.100561 5350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:51.100731 5350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:51.101325 5350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:51.102267 5350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:51.102315 5350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:51.102367 5350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:51.102396 5350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:51.111091 5350 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:41039
I20260806 18:43:51.111135 5553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:41039 every 8 connection(s)
I20260806 18:43:51.111989 5350 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:43:51.113548 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 5350
I20260806 18:43:51.113662 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:43:51.113976 9159 external_mini_cluster.cc:1428] Setting key ed095681247a8c4736e0034be2e6aa75
I20260806 18:43:51.115954 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37623
--builtin_ntp_servers=127.8.241.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:43:51.121820 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.113809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:37927 (local address 127.8.241.254:37623)
0806 18:43:51.113983 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:51.113988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:51.115174 (+ 1186us) server_negotiation.cc:407] Connection header received
0806 18:43:51.116679 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:51.116682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:51.116736 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:51.116810 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:51.117952 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.118535 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.119830 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.120147 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.120278 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.121318 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:51.121322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:51.121325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:51.121342 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:51.121367 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:51.121378 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:51.121473 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:51.121586 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 18:43:51.121655 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:43:51.122684 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.114254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37623 (local address 127.8.241.194:37927)
0806 18:43:51.115028 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:51.115080 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:51.116468 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:51.117023 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:51.117034 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:51.117128 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:51.117792 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.117811 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.118669 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.118673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.119713 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.119725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.120304 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.120307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.120455 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.121055 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:43:51.121057 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:51.121101 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:43:51.121169 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:51.121469 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:51.121474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:51.121478 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:43:51.121566 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 18:43:51.121793 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":121,"threads_started":1}
I20260806 18:43:51.123807 5554 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37623
I20260806 18:43:51.124040 5554 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:51.124547 5554 heartbeater.cc:511] Master 127.8.241.254:37623 requested a full tablet report, sending...
I20260806 18:43:51.125622 5072 ts_manager.cc:194] Registered new tserver with Master: 2176fa188ab849fd9fd74bdb3e5f0f69 (127.8.241.194:41039)
I20260806 18:43:51.126398 5072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.194:37927
W20260806 18:43:51.231683 5559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:51.231931 5559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:51.231967 5559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:51.235942 5559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:51.235998 5559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:51.236049 5559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:43:51.240617 5559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:37623
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:51.241624 5559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:51.243028 5559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:51.252310 5565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:51.252316 5567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:51.252316 5564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:51.252717 5559 server_base.cc:1034] running on GCE node
I20260806 18:43:51.253166 5559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:51.253796 5559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:51.254959 5559 hybrid_clock.cc:648] HybridClock initialized: now 1786041831254939 us; error 35 us; skew 500 ppm
I20260806 18:43:51.256724 5559 webserver.cc:460] Webserver started at http://127.8.241.195:35925/ using document root <none> and password file <none>
I20260806 18:43:51.257352 5559 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:51.257426 5559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:51.257629 5559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:51.259646 5559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "2b8db4a550c6423383c82ac1a2623620"
format_stamp: "Formatted at 2026-08-06 18:43:51 on dist-test-slave-k5rr"
server_key: "80f13026cb93447e3a41f80cb3b65b8f"
server_key_iv: "9c3ba178796a4892dcac0b8903e312f0"
server_key_version: "encryptionkey@0"
I20260806 18:43:51.260182 5559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "2b8db4a550c6423383c82ac1a2623620"
format_stamp: "Formatted at 2026-08-06 18:43:51 on dist-test-slave-k5rr"
server_key: "80f13026cb93447e3a41f80cb3b65b8f"
server_key_iv: "9c3ba178796a4892dcac0b8903e312f0"
server_key_version: "encryptionkey@0"
I20260806 18:43:51.263703 5559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260806 18:43:51.266122 5578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:51.267429 5559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 18:43:51.267534 5559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "2b8db4a550c6423383c82ac1a2623620"
format_stamp: "Formatted at 2026-08-06 18:43:51 on dist-test-slave-k5rr"
server_key: "80f13026cb93447e3a41f80cb3b65b8f"
server_key_iv: "9c3ba178796a4892dcac0b8903e312f0"
server_key_version: "encryptionkey@0"
I20260806 18:43:51.267635 5559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:51.289125 5559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:51.289783 5559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:51.289927 5559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:51.290483 5559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:51.291467 5559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:51.291514 5559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:51.291558 5559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:51.291584 5559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:51.300041 5559 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:40459
I20260806 18:43:51.300175 5748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:40459 every 8 connection(s)
I20260806 18:43:51.301128 5559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:43:51.303383 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 5559
I20260806 18:43:51.303488 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:43:51.303747 9159 external_mini_cluster.cc:1428] Setting key aadb1a0ce1b96e54106bd226999c71a5
I20260806 18:43:51.310509 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.303230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:44347 (local address 127.8.241.254:37623)
0806 18:43:51.303390 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:51.303395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:51.304564 (+ 1169us) server_negotiation.cc:407] Connection header received
0806 18:43:51.306121 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:51.306124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:51.306179 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:51.306260 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:51.307165 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.307689 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.308766 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.308987 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.309106 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.310062 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:51.310066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:51.310069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:51.310081 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:51.310099 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:51.310107 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:51.310165 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:51.310306 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 18:43:51.310370 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:43:51.311302 5751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.303659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:37623 (local address 127.8.241.195:44347)
0806 18:43:51.304417 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:51.304469 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:51.305916 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:51.306445 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:51.306452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:51.306535 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:51.307036 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.307048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.307807 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.307810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.308651 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.308660 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.309144 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.309148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.309321 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.309825 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:43:51.309827 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:51.309866 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:43:51.309926 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:51.310175 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:51.310185 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:51.310190 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:43:51.310299 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:43:51.310568 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":113,"threads_started":1}
I20260806 18:43:51.312266 5749 heartbeater.cc:347] Connected to a master server at 127.8.241.254:37623
I20260806 18:43:51.312485 5749 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:51.312959 5749 heartbeater.cc:511] Master 127.8.241.254:37623 requested a full tablet report, sending...
I20260806 18:43:51.313923 5072 ts_manager.cc:194] Registered new tserver with Master: 2b8db4a550c6423383c82ac1a2623620 (127.8.241.195:40459)
I20260806 18:43:51.314468 5072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.195:44347
I20260806 18:43:51.317345 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:43:51.322432 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.319186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44278 (local address 127.8.241.254:37623)
0806 18:43:51.319303 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:51.319307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:51.319484 (+ 177us) server_negotiation.cc:407] Connection header received
0806 18:43:51.319572 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:51.319574 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:51.319610 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:51.319671 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:51.319872 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.320358 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.321147 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.321355 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.321421 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.322072 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:51.322074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:51.322076 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:51.322086 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:51.322103 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:51.322110 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:51.322204 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:51.322279 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:43:51.322308 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:43:51.325001 5072 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:43:51.327944 5072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:43:51.341634 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.335725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:40459 (local address 127.0.0.1:56128)
0806 18:43:51.336453 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:51.336504 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:51.336628 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:51.337348 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:51.337354 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:51.337378 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:51.337509 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.337517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.338718 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.338721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.340031 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.340043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.341129 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.341133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.341261 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.341301 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:43:51.341410 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:43:51.341502 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"spinlock_wait_cycles":2176,"thread_start_us":65,"threads_started":1}
I20260806 18:43:51.341711 5766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.335882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56128 (local address 127.8.241.195:40459)
0806 18:43:51.336610 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:51.336622 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:51.336648 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:43:51.336814 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:51.336819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:51.336994 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:51.337159 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:51.337761 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.338628 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.340205 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.340994 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.341126 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.341315 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:51.341448 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 18:43:51.341551 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":160,"threads_started":1}
I20260806 18:43:51.341764 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.335889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:41039 (local address 127.0.0.1:47890)
0806 18:43:51.336453 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:51.336504 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:51.336628 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:51.337505 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:51.337507 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:51.337522 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:51.337613 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.337619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.338961 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.338964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.340042 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.340049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.341397 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.341400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.341488 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.341510 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:43:51.341584 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:43:51.341650 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"spinlock_wait_cycles":5376,"thread_start_us":107,"threads_started":1}
I20260806 18:43:51.341920 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.336039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47890 (local address 127.8.241.194:41039)
0806 18:43:51.336844 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:51.336867 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:51.336889 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:43:51.336969 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:51.336974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:51.337194 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:51.337355 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:51.337732 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.338844 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.340191 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.340994 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.341126 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.341315 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:51.341640 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 18:43:51.341758 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":99,"threads_started":1}
I20260806 18:43:51.342247 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.336603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:38361 (local address 127.0.0.1:43140)
0806 18:43:51.337326 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:51.337339 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:51.337430 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:51.338205 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:51.338208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:51.338229 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:51.338309 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.338314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.339417 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.339421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.340319 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.340329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.341905 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.341909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.341975 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.341998 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:43:51.342072 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:43:51.342135 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":99,"threads_started":1}
I20260806 18:43:51.342365 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.337058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.8.241.193:38361)
0806 18:43:51.337786 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:51.337798 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:51.337818 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:43:51.337890 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:51.337894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:51.338066 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:51.338219 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:51.338425 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.339315 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.340450 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.340995 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.341122 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.341319 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:51.342107 (+ 788us) server_negotiation.cc:299] Negotiation successful
0806 18:43:51.342202 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":117,"threads_started":1}
I20260806 18:43:51.344949 5450 tablet_service.cc:1467] Processing CreateTablet for tablet 63675649fe7b4086a4dc92414d99105b (DEFAULT_TABLE table=test-table [id=f7630e9cce1f43d9af6e618a90aa7540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:51.345089 5254 tablet_service.cc:1467] Processing CreateTablet for tablet 63675649fe7b4086a4dc92414d99105b (DEFAULT_TABLE table=test-table [id=f7630e9cce1f43d9af6e618a90aa7540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:51.345089 5648 tablet_service.cc:1467] Processing CreateTablet for tablet 63675649fe7b4086a4dc92414d99105b (DEFAULT_TABLE table=test-table [id=f7630e9cce1f43d9af6e618a90aa7540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:51.346047 5450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63675649fe7b4086a4dc92414d99105b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:51.346079 5254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63675649fe7b4086a4dc92414d99105b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:51.346079 5648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63675649fe7b4086a4dc92414d99105b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:51.352145 5775 tablet_bootstrap.cc:492] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: Bootstrap starting.
I20260806 18:43:51.353561 5776 tablet_bootstrap.cc:492] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: Bootstrap starting.
I20260806 18:43:51.353737 5777 tablet_bootstrap.cc:492] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e: Bootstrap starting.
I20260806 18:43:51.354866 5775 tablet_bootstrap.cc:654] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:51.355460 5777 tablet_bootstrap.cc:654] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:51.355717 5776 tablet_bootstrap.cc:654] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:51.355758 5775 log.cc:824] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:51.356137 5777 log.cc:824] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:51.356477 5776 log.cc:824] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:51.357745 5775 tablet_bootstrap.cc:492] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: No bootstrap required, opened a new log
I20260806 18:43:51.357936 5775 ts_tablet_manager.cc:1397] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260806 18:43:51.358155 5777 tablet_bootstrap.cc:492] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e: No bootstrap required, opened a new log
I20260806 18:43:51.358278 5776 tablet_bootstrap.cc:492] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: No bootstrap required, opened a new log
I20260806 18:43:51.358358 5777 ts_tablet_manager.cc:1397] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:43:51.358429 5776 ts_tablet_manager.cc:1397] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:43:51.361335 5775 raft_consensus.cc:348] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } }
I20260806 18:43:51.361549 5775 raft_consensus.cc:374] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:51.361603 5775 raft_consensus.cc:729] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2176fa188ab849fd9fd74bdb3e5f0f69, State: Initialized, Role: FOLLOWER
I20260806 18:43:51.362005 5775 consensus_queue.cc:260] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } }
I20260806 18:43:51.362619 5554 heartbeater.cc:503] Master 127.8.241.254:37623 was elected leader, sending a full tablet report...
I20260806 18:43:51.362475 5776 raft_consensus.cc:348] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } }
I20260806 18:43:51.362475 5777 raft_consensus.cc:348] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } }
I20260806 18:43:51.362710 5777 raft_consensus.cc:374] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:51.362722 5776 raft_consensus.cc:374] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:51.362762 5777 raft_consensus.cc:729] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fba2ba55f2a45fd9fcc400a2869644e, State: Initialized, Role: FOLLOWER
I20260806 18:43:51.362763 5776 raft_consensus.cc:729] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b8db4a550c6423383c82ac1a2623620, State: Initialized, Role: FOLLOWER
I20260806 18:43:51.362962 5775 ts_tablet_manager.cc:1428] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 18:43:51.363140 5555 tablet.cc:2369] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:51.363225 5777 consensus_queue.cc:260] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } }
I20260806 18:43:51.363214 5776 consensus_queue.cc:260] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } }
I20260806 18:43:51.364140 5749 heartbeater.cc:503] Master 127.8.241.254:37623 was elected leader, sending a full tablet report...
I20260806 18:43:51.364395 5777 ts_tablet_manager.cc:1428] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260806 18:43:51.364529 5345 heartbeater.cc:503] Master 127.8.241.254:37623 was elected leader, sending a full tablet report...
I20260806 18:43:51.364127 5776 ts_tablet_manager.cc:1428] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:43:51.389514 5787 raft_consensus.cc:482] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:51.389711 5787 raft_consensus.cc:504] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } }
I20260806 18:43:51.390923 5787 leader_election.cc:288] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b8db4a550c6423383c82ac1a2623620 (127.8.241.195:40459), 2176fa188ab849fd9fd74bdb3e5f0f69 (127.8.241.194:41039)
I20260806 18:43:51.394646 5348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.391106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:41039 (local address 127.8.241.193:46731)
0806 18:43:51.391243 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:51.391261 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:51.391354 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:51.391626 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:51.391628 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:51.391646 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:51.391771 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.391780 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.392742 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.392746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.393575 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.393585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.394247 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.394251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.394350 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.394377 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:43:51.394449 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:43:51.394518 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 18:43:51.394672 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.391240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:46731 (local address 127.8.241.194:41039)
0806 18:43:51.391373 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:51.391378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:51.391396 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:43:51.391469 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:51.391472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:51.391526 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:51.391612 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:51.391926 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.392632 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.393695 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.394106 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.394198 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.394238 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:51.394473 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 18:43:51.394547 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:43:51.394810 5789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.391106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:40459 (local address 127.8.241.193:50509)
0806 18:43:51.391621 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:51.391634 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:51.391743 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:51.391986 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:51.391989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:51.392008 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:51.392130 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.392137 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.392846 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.392848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.393677 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:51.393689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.394395 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.394398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:51.394490 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.394513 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:43:51.394585 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:43:51.394644 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":123,"thread_start_us":135,"threads_started":1}
I20260806 18:43:51.394819 5766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.391245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:50509 (local address 127.8.241.195:40459)
0806 18:43:51.391374 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:51.391378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:51.391723 (+ 345us) server_negotiation.cc:407] Connection header received
0806 18:43:51.391828 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:51.391832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:51.391887 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:51.391947 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:51.392235 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.392706 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.393826 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.394274 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.394396 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.394444 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:51.394615 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 18:43:51.394700 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:43:51.395366 5480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63675649fe7b4086a4dc92414d99105b" candidate_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" is_pre_election: true
I20260806 18:43:51.395467 5679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63675649fe7b4086a4dc92414d99105b" candidate_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b8db4a550c6423383c82ac1a2623620" is_pre_election: true
I20260806 18:43:51.395671 5480 raft_consensus.cc:2454] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fba2ba55f2a45fd9fcc400a2869644e in term 0.
I20260806 18:43:51.395747 5679 raft_consensus.cc:2454] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fba2ba55f2a45fd9fcc400a2869644e in term 0.
I20260806 18:43:51.396188 5175 leader_election.cc:302] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fba2ba55f2a45fd9fcc400a2869644e, 2b8db4a550c6423383c82ac1a2623620; no voters:
I20260806 18:43:51.396456 5787 raft_consensus.cc:2781] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:43:51.396548 5787 raft_consensus.cc:482] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:43:51.396595 5787 raft_consensus.cc:3037] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:51.397737 5787 raft_consensus.cc:504] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } }
I20260806 18:43:51.398090 5787 leader_election.cc:288] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [CANDIDATE]: Term 1 election: Requested vote from peers 2b8db4a550c6423383c82ac1a2623620 (127.8.241.195:40459), 2176fa188ab849fd9fd74bdb3e5f0f69 (127.8.241.194:41039)
I20260806 18:43:51.398368 5679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63675649fe7b4086a4dc92414d99105b" candidate_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b8db4a550c6423383c82ac1a2623620"
I20260806 18:43:51.398473 5679 raft_consensus.cc:3037] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:51.398628 5480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63675649fe7b4086a4dc92414d99105b" candidate_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69"
I20260806 18:43:51.398747 5480 raft_consensus.cc:3037] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:51.399721 5679 raft_consensus.cc:2454] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fba2ba55f2a45fd9fcc400a2869644e in term 1.
I20260806 18:43:51.399935 5480 raft_consensus.cc:2454] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fba2ba55f2a45fd9fcc400a2869644e in term 1.
I20260806 18:43:51.400045 5175 leader_election.cc:302] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fba2ba55f2a45fd9fcc400a2869644e, 2b8db4a550c6423383c82ac1a2623620; no voters:
I20260806 18:43:51.400226 5787 raft_consensus.cc:2781] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:51.400414 5787 raft_consensus.cc:686] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 1 LEADER]: Becoming Leader. State: Replica: 0fba2ba55f2a45fd9fcc400a2869644e, State: Running, Role: LEADER
I20260806 18:43:51.400743 5787 consensus_queue.cc:237] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } }
I20260806 18:43:51.403827 5072 catalog_manager.cc:5495] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fba2ba55f2a45fd9fcc400a2869644e (127.8.241.193). New cstate: current_term: 1 leader_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fba2ba55f2a45fd9fcc400a2869644e" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 38361 } health_report { overall_health: HEALTHY } } }
I20260806 18:43:51.421985 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.418428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.8.241.193:38361)
0806 18:43:51.418533 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:51.418537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:51.418569 (+ 32us) server_negotiation.cc:407] Connection header received
0806 18:43:51.418767 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:51.418770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:51.418842 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:51.418930 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:43:51.419237 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.419731 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.420621 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:51.420824 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:51.420900 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:51.421181 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:43:51.421580 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:43:51.421688 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:51.421838 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 18:43:51.421877 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 18:43:51.427093 5346 tablet.cc:2369] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:51.427672 5679 raft_consensus.cc:1261] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [term 1 FOLLOWER]: Refusing update from remote peer 0fba2ba55f2a45fd9fcc400a2869644e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:51.427701 5480 raft_consensus.cc:1261] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [term 1 FOLLOWER]: Refusing update from remote peer 0fba2ba55f2a45fd9fcc400a2869644e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:51.428339 5787 consensus_queue.cc:1035] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2176fa188ab849fd9fd74bdb3e5f0f69" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 41039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:51.428532 5790 consensus_queue.cc:1035] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b8db4a550c6423383c82ac1a2623620" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 40459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:51.437345 5798 mvcc.cc:205] Tried to move back new op lower bound from 7315627341520756736 to 7315627341420388352. Current Snapshot: MvccSnapshot[applied={T|T < 7315627341520756736}]
I20260806 18:43:51.437563 5797 mvcc.cc:205] Tried to move back new op lower bound from 7315627341520756736 to 7315627341420388352. Current Snapshot: MvccSnapshot[applied={T|T < 7315627341520756736}]
I20260806 18:43:51.443953 5071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:51.444092 5071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:51.446399 5071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1dae28d5a23b43fa8aef8453a95ced5d: Sending DeleteTablet for 3 replicas of tablet 63675649fe7b4086a4dc92414d99105b
I20260806 18:43:51.447120 5450 tablet_service.cc:1514] Processing DeleteTablet for tablet 63675649fe7b4086a4dc92414d99105b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:51 UTC) from {username='slave'} at 127.0.0.1:47890
I20260806 18:43:51.447165 5648 tablet_service.cc:1514] Processing DeleteTablet for tablet 63675649fe7b4086a4dc92414d99105b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:51 UTC) from {username='slave'} at 127.0.0.1:56128
I20260806 18:43:51.447199 5254 tablet_service.cc:1514] Processing DeleteTablet for tablet 63675649fe7b4086a4dc92414d99105b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:51 UTC) from {username='slave'} at 127.0.0.1:43140
I20260806 18:43:51.447603 5806 tablet_replica.cc:330] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: stopping tablet replica
I20260806 18:43:51.447593 5808 tablet_replica.cc:330] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e: stopping tablet replica
I20260806 18:43:51.447889 5808 raft_consensus.cc:2229] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:43:51.447889 5806 raft_consensus.cc:2229] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:51.448171 5806 raft_consensus.cc:2258] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:51.448190 5808 raft_consensus.cc:2258] T 63675649fe7b4086a4dc92414d99105b P 0fba2ba55f2a45fd9fcc400a2869644e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:51.448514 5809 tablet_replica.cc:330] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: stopping tablet replica
I20260806 18:43:51.448779 5809 raft_consensus.cc:2229] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:51.448848 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 5151
I20260806 18:43:51.449096 5809 raft_consensus.cc:2258] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:51.450263 5806 ts_tablet_manager.cc:1905] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:51.452414 5809 ts_tablet_manager.cc:1905] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:51.452948 5806 ts_tablet_manager.cc:1918] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:51.453068 5806 log.cc:1196] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/63675649fe7b4086a4dc92414d99105b
I20260806 18:43:51.453348 5806 ts_tablet_manager.cc:1939] T 63675649fe7b4086a4dc92414d99105b P 2176fa188ab849fd9fd74bdb3e5f0f69: Deleting consensus metadata
I20260806 18:43:51.454944 5809 ts_tablet_manager.cc:1918] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:51.455054 5809 log.cc:1196] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/63675649fe7b4086a4dc92414d99105b
I20260806 18:43:51.455377 5809 ts_tablet_manager.cc:1939] T 63675649fe7b4086a4dc92414d99105b P 2b8db4a550c6423383c82ac1a2623620: Deleting consensus metadata
I20260806 18:43:51.456882 5056 catalog_manager.cc:4841] TS 2b8db4a550c6423383c82ac1a2623620 (127.8.241.195:40459): tablet 63675649fe7b4086a4dc92414d99105b (table test-table [id=f7630e9cce1f43d9af6e618a90aa7540]) successfully deleted
I20260806 18:43:51.456964 5053 catalog_manager.cc:4841] TS 2176fa188ab849fd9fd74bdb3e5f0f69 (127.8.241.194:41039): tablet 63675649fe7b4086a4dc92414d99105b (table test-table [id=f7630e9cce1f43d9af6e618a90aa7540]) successfully deleted
W20260806 18:43:51.462855 5055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:38361 (error 108)
I20260806 18:43:51.463687 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 5350
I20260806 18:43:51.463910 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:51.463622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:38361 (local address 127.0.0.1:43148)
0806 18:43:51.463755 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:51.463825 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:38361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:43:51.481384 5055 catalog_manager.cc:4584] TS 0fba2ba55f2a45fd9fcc400a2869644e (127.8.241.193:38361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 63675649fe7b4086a4dc92414d99105b: Network error: Client connection negotiation failed: client connection to 127.8.241.193:38361: connect: Connection refused (error 111)
I20260806 18:43:51.488708 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 5559
I20260806 18:43:51.503221 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 5031
2026-08-06T18:43:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1024 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:43:51 dist-test-slave-k5rr krb5kdc[5814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:43:51 dist-test-slave-k5rr krb5kdc[5814](info): set up 2 sockets
Aug 06 18:43:51 dist-test-slave-k5rr krb5kdc[5814](info): commencing operation
krb5kdc: starting...
W20260806 18:43:52.632790 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:43:52 dist-test-slave-k5rr krb5kdc[5814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:43:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:43:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:43:52.754017 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:40919
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:45721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:40919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:52.864202 5837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:52.864439 5837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:52.864477 5837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:52.868284 5837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:52.868327 5837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:52.868345 5837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:52.868366 5837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:52.868378 5837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:52.873519 5837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:45721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:40919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:40919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:52.874434 5837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:52.875375 5837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:52.883715 5844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:52.883956 5837 server_base.cc:1034] running on GCE node
W20260806 18:43:52.883735 5846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:52.883775 5843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:52.884598 5837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:52.885731 5837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:52.886898 5837 hybrid_clock.cc:648] HybridClock initialized: now 1786041832886876 us; error 37 us; skew 500 ppm
Aug 06 18:43:52 dist-test-slave-k5rr krb5kdc[5814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041832, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:52.899106 5837 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:43:52.900036 5837 webserver.cc:460] Webserver started at http://127.8.241.254:39825/ using document root <none> and password file <none>
I20260806 18:43:52.900586 5837 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:52.900647 5837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:52.900838 5837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:52.902933 5837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "324855646489441e9d736ba03c5ac49f"
format_stamp: "Formatted at 2026-08-06 18:43:52 on dist-test-slave-k5rr"
server_key: "979c991bdc431e4747bf7775d124f227"
server_key_iv: "1c89137d7af04f75106b3903045f054b"
server_key_version: "encryptionkey@0"
I20260806 18:43:52.903491 5837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "324855646489441e9d736ba03c5ac49f"
format_stamp: "Formatted at 2026-08-06 18:43:52 on dist-test-slave-k5rr"
server_key: "979c991bdc431e4747bf7775d124f227"
server_key_iv: "1c89137d7af04f75106b3903045f054b"
server_key_version: "encryptionkey@0"
I20260806 18:43:52.907136 5837 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 18:43:52.909556 5857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:52.910737 5837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:52.910840 5837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "324855646489441e9d736ba03c5ac49f"
format_stamp: "Formatted at 2026-08-06 18:43:52 on dist-test-slave-k5rr"
server_key: "979c991bdc431e4747bf7775d124f227"
server_key_iv: "1c89137d7af04f75106b3903045f054b"
server_key_version: "encryptionkey@0"
I20260806 18:43:52.910952 5837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:52.943063 5837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:52.948153 5837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:52.948374 5837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:52.956681 5837 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:40919
I20260806 18:43:52.956712 5933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:40919 every 8 connection(s)
I20260806 18:43:52.957983 5837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:52.960309 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 5837
I20260806 18:43:52.960456 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:43:52.960754 9159 external_mini_cluster.cc:1428] Setting key bdb6b331f669346d6d955d5ffb0ed80d
I20260806 18:43:52.961611 5934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:52.969312 5934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f: Bootstrap starting.
Aug 06 18:43:52 dist-test-slave-k5rr krb5kdc[5814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:52.972271 5934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:52.973331 5934 log.cc:824] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:52.975840 5934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f: No bootstrap required, opened a new log
I20260806 18:43:52.977797 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:52.962255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.8.241.254:40919)
0806 18:43:52.962722 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:52.962742 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:52.962780 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:43:52.963503 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:52.963526 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:52.963866 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:52.964136 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:52.965220 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:52.966251 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:52.967477 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:52.967963 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:52.968108 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:52.971366 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:52.971373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:52.971390 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:52.971427 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:52.973759 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:52.974350 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:52.974357 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:52.974363 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:52.974509 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:52.974870 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:52.974874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:52.974875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:52.975414 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:52.975620 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:52.976460 (+ 840us) server_negotiation.cc:299] Negotiation successful
0806 18:43:52.976793 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":135,"threads_started":1}
I20260806 18:43:52.980499 5934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324855646489441e9d736ba03c5ac49f" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 40919 } }
I20260806 18:43:52.980743 5934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:52.980808 5934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 324855646489441e9d736ba03c5ac49f, State: Initialized, Role: FOLLOWER
I20260806 18:43:52.981276 5934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324855646489441e9d736ba03c5ac49f" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 40919 } }
I20260806 18:43:52.981426 5934 raft_consensus.cc:388] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:52.981496 5934 raft_consensus.cc:482] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:52.981598 5934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:52.982569 5934 raft_consensus.cc:504] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324855646489441e9d736ba03c5ac49f" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 40919 } }
I20260806 18:43:52.982923 5934 leader_election.cc:302] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 324855646489441e9d736ba03c5ac49f; no voters:
I20260806 18:43:52.983263 5934 leader_election.cc:288] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:43:52.983385 5939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:52.983618 5939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [term 1 LEADER]: Becoming Leader. State: Replica: 324855646489441e9d736ba03c5ac49f, State: Running, Role: LEADER
I20260806 18:43:52.983944 5939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324855646489441e9d736ba03c5ac49f" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 40919 } }
I20260806 18:43:52.984613 5934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:52.985749 5943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 324855646489441e9d736ba03c5ac49f. Latest consensus state: current_term: 1 leader_uuid: "324855646489441e9d736ba03c5ac49f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324855646489441e9d736ba03c5ac49f" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 40919 } } }
I20260806 18:43:52.985749 5942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "324855646489441e9d736ba03c5ac49f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324855646489441e9d736ba03c5ac49f" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 40919 } } }
I20260806 18:43:52.985915 5943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:52.985920 5942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:52.986359 5952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:52.989636 5952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:52.995428 5952 catalog_manager.cc:1347] Generated new cluster ID: 8348082d57c848e58e9547e62591688d
I20260806 18:43:52.995507 5952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:53.006064 5952 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:43:53.007018 5952 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:53.016305 5952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f: Generated new TSK 0
I20260806 18:43:53.017117 5952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.241.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
Entry for principal kudu/127.8.241.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab.
I20260806 18:43:53.051576 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:40919
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:45721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:53.184588 5969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:53.184872 5969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:53.184926 5969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:53.189209 5969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:53.189273 5969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:53.189343 5969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:43:53.194648 5969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:45721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.193.keytab
--principal=kudu/127.8.241.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:53.195703 5969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:53.196749 5969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:53.206326 5977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:53.206383 5969 server_base.cc:1034] running on GCE node
W20260806 18:43:53.206394 5979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:53.206317 5982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:53.206902 5969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:53.207544 5969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:53.208791 5969 hybrid_clock.cc:648] HybridClock initialized: now 1786041833208766 us; error 34 us; skew 500 ppm
Aug 06 18:43:53 dist-test-slave-k5rr krb5kdc[5814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041833, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:53.211812 5969 init.cc:377] Logged in from keytab as kudu/127.8.241.193@KRBTEST.COM (short username kudu)
I20260806 18:43:53.212813 5969 webserver.cc:460] Webserver started at http://127.8.241.193:46359/ using document root <none> and password file <none>
I20260806 18:43:53.213438 5969 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:53.213500 5969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:53.213734 5969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:53.215946 5969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "e92daca9e65b4166b8a166c66027f967"
format_stamp: "Formatted at 2026-08-06 18:43:53 on dist-test-slave-k5rr"
server_key: "61a2f4d52d3b3840f9b092ef28925b9a"
server_key_iv: "c1cc9435f9150a38bef7a693ecab1316"
server_key_version: "encryptionkey@0"
I20260806 18:43:53.216497 5969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "e92daca9e65b4166b8a166c66027f967"
format_stamp: "Formatted at 2026-08-06 18:43:53 on dist-test-slave-k5rr"
server_key: "61a2f4d52d3b3840f9b092ef28925b9a"
server_key_iv: "c1cc9435f9150a38bef7a693ecab1316"
server_key_version: "encryptionkey@0"
I20260806 18:43:53.220304 5969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:43:53.222980 5990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:53.224262 5969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:43:53.224370 5969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "e92daca9e65b4166b8a166c66027f967"
format_stamp: "Formatted at 2026-08-06 18:43:53 on dist-test-slave-k5rr"
server_key: "61a2f4d52d3b3840f9b092ef28925b9a"
server_key_iv: "c1cc9435f9150a38bef7a693ecab1316"
server_key_version: "encryptionkey@0"
I20260806 18:43:53.224483 5969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:53.261344 5969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:53.264411 5969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:53.264609 5969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:53.265291 5969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:53.266187 5969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:53.266233 5969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:53.266275 5969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:53.266305 5969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:53.275136 5969 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:34119
I20260806 18:43:53.275177 6163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:34119 every 8 connection(s)
I20260806 18:43:53.276015 5969 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:43:53.279346 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 5969
I20260806 18:43:53.279498 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:43:53.279827 9159 external_mini_cluster.cc:1428] Setting key 4b88deff0711126ad39ab8c502b871b0
Aug 06 18:43:53 dist-test-slave-k5rr krb5kdc[5814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041833, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.193@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.194@KRBTEST.COM; defaulting to no policy
I20260806 18:43:53.290710 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.277849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:47047 (local address 127.8.241.254:40919)
0806 18:43:53.277995 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.278000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.278871 (+ 871us) server_negotiation.cc:407] Connection header received
0806 18:43:53.280560 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.280562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.280614 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.280695 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:53.282062 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.282604 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.283765 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.284014 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.284111 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.287112 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:53.287115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:53.287117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:53.287142 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:53.288461 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:53.289130 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:53.289133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:53.289134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:53.289205 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:53.289579 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:53.289581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:53.289582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:53.289748 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:53.289818 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:53.290445 (+ 627us) server_negotiation.cc:299] Negotiation successful
0806 18:43:53.290550 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:43:53.291651 6170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.278204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:40919 (local address 127.8.241.193:47047)
0806 18:43:53.278680 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:53.278753 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:53.280318 (+ 1565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:53.280878 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:53.280888 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:53.281395 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:53.281905 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.281921 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.282746 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.282749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.283636 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.283647 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.284166 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.284171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.284378 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.285040 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:53.285064 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:53.286937 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:53.288585 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:53.288592 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:53.288618 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:53.288995 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:53.289294 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:53.289297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:53.289299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:53.289490 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:53.289853 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:53.289861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:53.290195 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 18:43:53.290416 (+ 221us) client_negotiation.cc:240] Negotiation successful
0806 18:43:53.290780 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":121,"threads_started":1}
I20260806 18:43:53.292845 6164 heartbeater.cc:347] Connected to a master server at 127.8.241.254:40919
I20260806 18:43:53.293140 6164 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.194@KRBTEST.COM" created.
I20260806 18:43:53.293665 6164 heartbeater.cc:511] Master 127.8.241.254:40919 requested a full tablet report, sending...
I20260806 18:43:53.295328 5882 ts_manager.cc:194] Registered new tserver with Master: e92daca9e65b4166b8a166c66027f967 (127.8.241.193:34119)
I20260806 18:43:53.296446 5882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.193@KRBTEST.COM'} at 127.8.241.193:47047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
Entry for principal kudu/127.8.241.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab.
I20260806 18:43:53.309285 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:40919
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:45721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:53.424263 6173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:53.424520 6173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:53.424563 6173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:53.428817 6173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:53.428884 6173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:53.428946 6173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:43:53.434166 6173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:45721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.194.keytab
--principal=kudu/127.8.241.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:53.435176 6173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:53.436153 6173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:53.445492 6179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:53.445531 6183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:53.445519 6180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:53.445657 6173 server_base.cc:1034] running on GCE node
I20260806 18:43:53.446177 6173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:53.446889 6173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:53.448065 6173 hybrid_clock.cc:648] HybridClock initialized: now 1786041833448045 us; error 32 us; skew 500 ppm
Aug 06 18:43:53 dist-test-slave-k5rr krb5kdc[5814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041833, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:53.450804 6173 init.cc:377] Logged in from keytab as kudu/127.8.241.194@KRBTEST.COM (short username kudu)
I20260806 18:43:53.451756 6173 webserver.cc:460] Webserver started at http://127.8.241.194:40823/ using document root <none> and password file <none>
I20260806 18:43:53.452312 6173 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:53.452379 6173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:53.452575 6173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:53.454640 6173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "dc9455da4a0b44b89b7d3dd88d4ca312"
format_stamp: "Formatted at 2026-08-06 18:43:53 on dist-test-slave-k5rr"
server_key: "1af1f8a95648498ffa7cc9f02c572b73"
server_key_iv: "cbdc00074a8e3006d8d79af56406974c"
server_key_version: "encryptionkey@0"
I20260806 18:43:53.455227 6173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "dc9455da4a0b44b89b7d3dd88d4ca312"
format_stamp: "Formatted at 2026-08-06 18:43:53 on dist-test-slave-k5rr"
server_key: "1af1f8a95648498ffa7cc9f02c572b73"
server_key_iv: "cbdc00074a8e3006d8d79af56406974c"
server_key_version: "encryptionkey@0"
I20260806 18:43:53.459074 6173 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 18:43:53.461455 6194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:53.462610 6173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:53.462718 6173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "dc9455da4a0b44b89b7d3dd88d4ca312"
format_stamp: "Formatted at 2026-08-06 18:43:53 on dist-test-slave-k5rr"
server_key: "1af1f8a95648498ffa7cc9f02c572b73"
server_key_iv: "cbdc00074a8e3006d8d79af56406974c"
server_key_version: "encryptionkey@0"
I20260806 18:43:53.462812 6173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:53.491235 6173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:53.494484 6173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:53.494669 6173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:53.495294 6173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:53.496176 6173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:53.496234 6173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:53.496291 6173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:53.496315 6173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:53.504803 6173 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:42703
I20260806 18:43:53.504858 6372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:42703 every 8 connection(s)
I20260806 18:43:53.505748 6173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:43:53.506057 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 6173
I20260806 18:43:53.506179 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:43:53.506515 9159 external_mini_cluster.cc:1428] Setting key 30dbd2837c6263a5d056e3da067d0159
Aug 06 18:43:53 dist-test-slave-k5rr krb5kdc[5814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041833, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.194@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.195@KRBTEST.COM" created.
I20260806 18:43:53.520807 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.507823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:39341 (local address 127.8.241.254:40919)
0806 18:43:53.507955 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.507960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.508891 (+ 931us) server_negotiation.cc:407] Connection header received
0806 18:43:53.510292 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.510296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.510366 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.510437 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:53.511731 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.512266 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.513985 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.514244 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.514336 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.517317 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:53.517321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:53.517323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:53.517348 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:53.518716 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:53.519239 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:53.519242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:53.519243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:53.519327 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:53.519635 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:53.519637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:53.519639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:53.519828 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:53.519922 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:53.520508 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 18:43:53.520630 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:43:53.521687 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.508238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:40919 (local address 127.8.241.194:39341)
0806 18:43:53.508751 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:53.508795 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:53.510089 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:53.510631 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:53.510638 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:53.511002 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:53.511573 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.511593 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.512398 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.512402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.513849 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.513873 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.514418 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.514423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.514635 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.515301 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:53.515323 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:53.517122 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:53.518827 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:53.518832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:53.518845 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:53.519114 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:53.519395 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:53.519397 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:53.519399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:53.519525 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:53.519936 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:53.519944 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:53.520216 (+ 272us) client_negotiation.cc:769] Sending connection context
0806 18:43:53.520430 (+ 214us) client_negotiation.cc:240] Negotiation successful
0806 18:43:53.520762 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":138,"threads_started":1}
I20260806 18:43:53.522807 6373 heartbeater.cc:347] Connected to a master server at 127.8.241.254:40919
I20260806 18:43:53.523041 6373 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:53.523550 6373 heartbeater.cc:511] Master 127.8.241.254:40919 requested a full tablet report, sending...
I20260806 18:43:53.524600 5882 ts_manager.cc:194] Registered new tserver with Master: dc9455da4a0b44b89b7d3dd88d4ca312 (127.8.241.194:42703)
I20260806 18:43:53.525207 5882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.194@KRBTEST.COM'} at 127.8.241.194:39341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
Entry for principal kudu/127.8.241.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab.
I20260806 18:43:53.536480 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:40919
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:45721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:53.648794 6380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:53.649071 6380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:53.649137 6380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:53.653111 6380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:53.653165 6380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:53.653244 6380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:43:53.658363 6380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:45721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.195.keytab
--principal=kudu/127.8.241.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:53.659446 6380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:53.660504 6380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:53.670202 6387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:53.670212 6392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:53.670308 6380 server_base.cc:1034] running on GCE node
W20260806 18:43:53.670204 6390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:53.670902 6380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:53.671525 6380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:53.672701 6380 hybrid_clock.cc:648] HybridClock initialized: now 1786041833672682 us; error 29 us; skew 500 ppm
Aug 06 18:43:53 dist-test-slave-k5rr krb5kdc[5814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041833, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:53.675422 6380 init.cc:377] Logged in from keytab as kudu/127.8.241.195@KRBTEST.COM (short username kudu)
I20260806 18:43:53.676290 6380 webserver.cc:460] Webserver started at http://127.8.241.195:37631/ using document root <none> and password file <none>
I20260806 18:43:53.676807 6380 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:53.676864 6380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:53.677093 6380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:53.679085 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "a37a49200c5c4381b9d1f819c9a66fb0"
format_stamp: "Formatted at 2026-08-06 18:43:53 on dist-test-slave-k5rr"
server_key: "97652f131ba3ad19e8c0472e446e2b7a"
server_key_iv: "bc8e1261c18ff2145f4be68c3dc0c663"
server_key_version: "encryptionkey@0"
I20260806 18:43:53.679600 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "a37a49200c5c4381b9d1f819c9a66fb0"
format_stamp: "Formatted at 2026-08-06 18:43:53 on dist-test-slave-k5rr"
server_key: "97652f131ba3ad19e8c0472e446e2b7a"
server_key_iv: "bc8e1261c18ff2145f4be68c3dc0c663"
server_key_version: "encryptionkey@0"
I20260806 18:43:53.683377 6380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:43:53.685945 6401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:53.687242 6380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:53.687341 6380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "a37a49200c5c4381b9d1f819c9a66fb0"
format_stamp: "Formatted at 2026-08-06 18:43:53 on dist-test-slave-k5rr"
server_key: "97652f131ba3ad19e8c0472e446e2b7a"
server_key_iv: "bc8e1261c18ff2145f4be68c3dc0c663"
server_key_version: "encryptionkey@0"
I20260806 18:43:53.687435 6380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:53.701208 6380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:53.704042 6380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:53.704223 6380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:53.704808 6380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:53.705700 6380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:53.705749 6380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:53.705804 6380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:53.705833 6380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:53.714270 6380 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:37031
I20260806 18:43:53.714304 6568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:37031 every 8 connection(s)
I20260806 18:43:53.715167 6380 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:43:53.722855 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 6380
I20260806 18:43:53.722997 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:43:53.723307 9159 external_mini_cluster.cc:1428] Setting key bd4f053931898733c2ea6d046e440150
Aug 06 18:43:53 dist-test-slave-k5rr krb5kdc[5814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041833, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.195@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:43:53.728652 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.716922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:32945 (local address 127.8.241.254:40919)
0806 18:43:53.717107 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.717112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.717942 (+ 830us) server_negotiation.cc:407] Connection header received
0806 18:43:53.719225 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.719227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.719273 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.719345 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:53.720472 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.720988 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.722137 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.722386 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.722483 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.725256 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:53.725259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:53.725261 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:53.725290 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:53.726481 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:53.727006 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:53.727011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:53.727012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:53.727110 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:53.727448 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:53.727452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:53.727454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:53.727650 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:53.727751 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:53.728330 (+ 579us) server_negotiation.cc:299] Negotiation successful
0806 18:43:53.728471 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:43:53.729545 6571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.717352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:40919 (local address 127.8.241.195:32945)
0806 18:43:53.717795 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:53.717838 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:53.719038 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:53.719513 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:53.719520 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:53.719861 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:53.720342 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.720367 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.721112 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.721114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.721994 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.722004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.722534 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.722537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.722701 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.723293 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:43:53.723309 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:53.725060 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:53.726601 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:53.726607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:53.726624 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:53.726901 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:53.727201 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:43:53.727204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:43:53.727206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:43:53.727352 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:43:53.727764 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:53.727772 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:53.728078 (+ 306us) client_negotiation.cc:769] Sending connection context
0806 18:43:53.728264 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 18:43:53.728586 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":101,"threads_started":1}
I20260806 18:43:53.730588 6569 heartbeater.cc:347] Connected to a master server at 127.8.241.254:40919
I20260806 18:43:53.730818 6569 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:53.731293 6569 heartbeater.cc:511] Master 127.8.241.254:40919 requested a full tablet report, sending...
I20260806 18:43:53.732290 5882 ts_manager.cc:194] Registered new tserver with Master: a37a49200c5c4381b9d1f819c9a66fb0 (127.8.241.195:37031)
I20260806 18:43:53.732823 5882 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.241.195@KRBTEST.COM'} at 127.8.241.195:32945
I20260806 18:43:53.737164 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:43:53.745849 5937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.739098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.8.241.254:40919)
0806 18:43:53.739216 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.739220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.739399 (+ 179us) server_negotiation.cc:407] Connection header received
0806 18:43:53.739499 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.739501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.739537 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.739598 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:53.740309 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.740800 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.741652 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.741859 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.741931 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.743108 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:53.743111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:53.743113 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:53.743135 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:53.744348 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:53.744727 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:53.744729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:53.744730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:53.744795 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:53.745108 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:53.745110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:53.745112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:53.745323 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:53.745436 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:53.745629 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 18:43:53.745727 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:43:53.748579 5882 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:43:53.751655 5882 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:43:53.765334 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.759915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37031 (local address 127.0.0.1:36452)
0806 18:43:53.760542 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:53.760556 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:53.760689 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:53.761529 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:53.761535 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:53.761567 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:53.761788 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.761796 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.762913 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.762915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.764012 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.764022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.764788 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.764791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.764907 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.764947 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:43:53.765064 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 18:43:53.765185 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":35,"spinlock_wait_cycles":19712,"thread_start_us":105,"threads_started":1}
I20260806 18:43:53.765340 6594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.760288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36452 (local address 127.8.241.195:37031)
0806 18:43:53.761076 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.761088 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.761111 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:43:53.761187 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.761192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.761374 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.761529 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:53.761913 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.762789 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.764127 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.764670 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.764764 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.764916 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:53.765118 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 18:43:53.765209 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":155,"threads_started":1}
I20260806 18:43:53.766114 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.759748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:42703 (local address 127.0.0.1:52848)
0806 18:43:53.760501 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:53.760551 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:53.760690 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:53.761529 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:53.761535 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:53.761566 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:53.761827 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.761834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.763291 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.763294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.764448 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.764458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.765728 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.765732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.765825 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.765848 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:43:53.765908 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:43:53.765975 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":81,"spinlock_wait_cycles":19072,"thread_start_us":84,"threads_started":1}
I20260806 18:43:53.766346 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.761427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.8.241.193:34119)
0806 18:43:53.761870 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.761886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.761907 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:43:53.761976 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.761981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.762166 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.762315 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:53.762527 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.763581 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.764972 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.765763 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.765861 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.765993 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:53.766106 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 18:43:53.766198 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":74,"threads_started":1}
I20260806 18:43:53.766573 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.760986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34119 (local address 127.0.0.1:54782)
0806 18:43:53.761246 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:53.761259 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:53.761333 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:53.762308 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:53.762311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:53.762330 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:53.762416 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.762422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.763746 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.763750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.764842 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.764852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.765880 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.765884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.765971 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.765995 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:43:53.766079 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:43:53.766142 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260806 18:43:53.766625 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.759900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52848 (local address 127.8.241.194:42703)
0806 18:43:53.760247 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.760258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.760830 (+ 572us) server_negotiation.cc:407] Connection header received
0806 18:43:53.760917 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.760922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.761176 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.761333 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:53.761952 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.763134 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.764583 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.765424 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.765522 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.765660 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:53.766403 (+ 743us) server_negotiation.cc:299] Negotiation successful
0806 18:43:53.766499 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":120,"threads_started":1}
I20260806 18:43:53.768976 6479 tablet_service.cc:1467] Processing CreateTablet for tablet a0630b0fbaf24dc6a7c1abd9a021348e (DEFAULT_TABLE table=test-table [id=4490014e603b4d2482b0bb5f2570751c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:53.769366 6271 tablet_service.cc:1467] Processing CreateTablet for tablet a0630b0fbaf24dc6a7c1abd9a021348e (DEFAULT_TABLE table=test-table [id=4490014e603b4d2482b0bb5f2570751c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:53.770201 6479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0630b0fbaf24dc6a7c1abd9a021348e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:53.770303 6271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0630b0fbaf24dc6a7c1abd9a021348e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:53.771289 6072 tablet_service.cc:1467] Processing CreateTablet for tablet a0630b0fbaf24dc6a7c1abd9a021348e (DEFAULT_TABLE table=test-table [id=4490014e603b4d2482b0bb5f2570751c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:53.772097 6072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0630b0fbaf24dc6a7c1abd9a021348e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:53.777947 6599 tablet_bootstrap.cc:492] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: Bootstrap starting.
I20260806 18:43:53.778506 6598 tablet_bootstrap.cc:492] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: Bootstrap starting.
I20260806 18:43:53.780123 6598 tablet_bootstrap.cc:654] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:53.780463 6599 tablet_bootstrap.cc:654] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:53.781193 6598 log.cc:824] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:53.781225 6599 log.cc:824] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:53.781426 6600 tablet_bootstrap.cc:492] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967: Bootstrap starting.
I20260806 18:43:53.783187 6599 tablet_bootstrap.cc:492] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: No bootstrap required, opened a new log
I20260806 18:43:53.783193 6598 tablet_bootstrap.cc:492] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: No bootstrap required, opened a new log
I20260806 18:43:53.783356 6600 tablet_bootstrap.cc:654] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:53.783385 6598 ts_tablet_manager.cc:1397] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:43:53.783382 6599 ts_tablet_manager.cc:1397] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260806 18:43:53.784320 6600 log.cc:824] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:53.786156 6600 tablet_bootstrap.cc:492] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967: No bootstrap required, opened a new log
I20260806 18:43:53.786376 6600 ts_tablet_manager.cc:1397] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:43:53.787221 6599 raft_consensus.cc:348] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } }
I20260806 18:43:53.787221 6598 raft_consensus.cc:348] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } }
I20260806 18:43:53.787503 6599 raft_consensus.cc:374] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:53.787503 6598 raft_consensus.cc:374] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:53.787563 6599 raft_consensus.cc:729] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9455da4a0b44b89b7d3dd88d4ca312, State: Initialized, Role: FOLLOWER
I20260806 18:43:53.787563 6598 raft_consensus.cc:729] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a37a49200c5c4381b9d1f819c9a66fb0, State: Initialized, Role: FOLLOWER
I20260806 18:43:53.788012 6598 consensus_queue.cc:260] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } }
I20260806 18:43:53.788012 6599 consensus_queue.cc:260] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } }
I20260806 18:43:53.788664 6569 heartbeater.cc:503] Master 127.8.241.254:40919 was elected leader, sending a full tablet report...
I20260806 18:43:53.788964 6373 heartbeater.cc:503] Master 127.8.241.254:40919 was elected leader, sending a full tablet report...
I20260806 18:43:53.789033 6599 ts_tablet_manager.cc:1428] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260806 18:43:53.789597 6600 raft_consensus.cc:348] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } }
I20260806 18:43:53.789834 6600 raft_consensus.cc:374] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:53.789886 6600 raft_consensus.cc:729] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e92daca9e65b4166b8a166c66027f967, State: Initialized, Role: FOLLOWER
I20260806 18:43:53.790603 6600 consensus_queue.cc:260] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } }
I20260806 18:43:53.791028 6598 ts_tablet_manager.cc:1428] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 18:43:53.791299 6164 heartbeater.cc:503] Master 127.8.241.254:40919 was elected leader, sending a full tablet report...
I20260806 18:43:53.791589 6600 ts_tablet_manager.cc:1428] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:43:53.952654 6607 raft_consensus.cc:482] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:53.952874 6607 raft_consensus.cc:504] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } }
I20260806 18:43:53.954037 6607 leader_election.cc:288] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a37a49200c5c4381b9d1f819c9a66fb0 (127.8.241.195:37031), e92daca9e65b4166b8a166c66027f967 (127.8.241.193:34119)
I20260806 18:43:53.958591 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.954303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:37031 (local address 127.8.241.194:59583)
0806 18:43:53.955032 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:53.955045 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:53.955136 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:53.955359 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:53.955361 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:53.955373 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:53.955466 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.955470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.956310 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.956313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.957505 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.957512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.958163 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.958167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.958263 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.958289 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:43:53.958363 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:43:53.958434 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":346,"thread_start_us":157,"threads_started":1}
I20260806 18:43:53.958613 6594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.954411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:59583 (local address 127.8.241.195:37031)
0806 18:43:53.954538 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.954542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.955153 (+ 611us) server_negotiation.cc:407] Connection header received
0806 18:43:53.955218 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.955221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.955272 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.955356 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:53.955564 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.956209 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.957621 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.958047 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.958158 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.958216 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:53.958392 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 18:43:53.958480 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:43:53.958853 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.954411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:39195 (local address 127.8.241.193:34119)
0806 18:43:53.954558 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.954563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.954591 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:43:53.954663 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.954666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.954719 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.954798 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:53.955173 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.955943 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.957442 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.958104 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.958240 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.958300 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:53.958649 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 18:43:53.958734 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:43:53.959075 6375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.954303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34119 (local address 127.8.241.194:39195)
0806 18:43:53.954425 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:53.954441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:53.954543 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:53.954854 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:53.954858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:53.954886 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:53.955011 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.955018 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.956054 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.956056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.957300 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:53.957312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.958285 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.958288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:53.958404 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:53.958423 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:43:53.958485 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:43:53.958539 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":21}
I20260806 18:43:53.959439 6508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0630b0fbaf24dc6a7c1abd9a021348e" candidate_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" is_pre_election: true
I20260806 18:43:53.959713 6508 raft_consensus.cc:2454] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9455da4a0b44b89b7d3dd88d4ca312 in term 0.
I20260806 18:43:53.959725 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0630b0fbaf24dc6a7c1abd9a021348e" candidate_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e92daca9e65b4166b8a166c66027f967" is_pre_election: true
I20260806 18:43:53.960019 6097 raft_consensus.cc:2454] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc9455da4a0b44b89b7d3dd88d4ca312 in term 0.
I20260806 18:43:53.960187 6198 leader_election.cc:302] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a37a49200c5c4381b9d1f819c9a66fb0, dc9455da4a0b44b89b7d3dd88d4ca312; no voters:
I20260806 18:43:53.960489 6607 raft_consensus.cc:2781] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:43:53.960558 6607 raft_consensus.cc:482] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:43:53.960601 6607 raft_consensus.cc:3037] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:53.961686 6607 raft_consensus.cc:504] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } }
I20260806 18:43:53.962025 6607 leader_election.cc:288] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [CANDIDATE]: Term 1 election: Requested vote from peers a37a49200c5c4381b9d1f819c9a66fb0 (127.8.241.195:37031), e92daca9e65b4166b8a166c66027f967 (127.8.241.193:34119)
I20260806 18:43:53.962343 6508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0630b0fbaf24dc6a7c1abd9a021348e" candidate_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a37a49200c5c4381b9d1f819c9a66fb0"
I20260806 18:43:53.962450 6097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0630b0fbaf24dc6a7c1abd9a021348e" candidate_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e92daca9e65b4166b8a166c66027f967"
I20260806 18:43:53.962478 6508 raft_consensus.cc:3037] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:53.962571 6097 raft_consensus.cc:3037] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:53.963735 6508 raft_consensus.cc:2454] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9455da4a0b44b89b7d3dd88d4ca312 in term 1.
I20260806 18:43:53.963735 6097 raft_consensus.cc:2454] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc9455da4a0b44b89b7d3dd88d4ca312 in term 1.
I20260806 18:43:53.964046 6198 leader_election.cc:302] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a37a49200c5c4381b9d1f819c9a66fb0, dc9455da4a0b44b89b7d3dd88d4ca312; no voters:
I20260806 18:43:53.964197 6607 raft_consensus.cc:2781] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:53.964403 6607 raft_consensus.cc:686] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 1 LEADER]: Becoming Leader. State: Replica: dc9455da4a0b44b89b7d3dd88d4ca312, State: Running, Role: LEADER
I20260806 18:43:53.964737 6607 consensus_queue.cc:237] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } }
W20260806 18:43:53.966073 6570 tablet.cc:2369] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:53.968101 5882 catalog_manager.cc:5495] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc9455da4a0b44b89b7d3dd88d4ca312 (127.8.241.194). New cstate: current_term: 1 leader_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc9455da4a0b44b89b7d3dd88d4ca312" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 42703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 } health_report { overall_health: UNKNOWN } } }
I20260806 18:43:54.000888 6591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:53.997399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.8.241.194:42703)
0806 18:43:53.997532 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:53.997537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:53.997552 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:43:53.997639 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:53.997642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:53.997698 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:53.997837 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:43:53.998091 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.998593 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.999433 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:53.999656 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:53.999766 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.000044 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:43:54.000480 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:43:54.000594 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:54.000665 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 18:43:54.000732 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:43:54.007076 6508 raft_consensus.cc:1261] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [term 1 FOLLOWER]: Refusing update from remote peer dc9455da4a0b44b89b7d3dd88d4ca312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:54.007085 6097 raft_consensus.cc:1261] T a0630b0fbaf24dc6a7c1abd9a021348e P e92daca9e65b4166b8a166c66027f967 [term 1 FOLLOWER]: Refusing update from remote peer dc9455da4a0b44b89b7d3dd88d4ca312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 18:43:54.007149 6374 tablet.cc:2369] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:54.007858 6611 consensus_queue.cc:1035] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e92daca9e65b4166b8a166c66027f967" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 34119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:54.008041 6607 consensus_queue.cc:1035] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a37a49200c5c4381b9d1f819c9a66fb0" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 37031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:54.015067 6614 mvcc.cc:205] Tried to move back new op lower bound from 7315627352085483520 to 7315627351922565120. Current Snapshot: MvccSnapshot[applied={T|T < 7315627352085483520}]
I20260806 18:43:54.019258 6619 mvcc.cc:205] Tried to move back new op lower bound from 7315627352085483520 to 7315627351922565120. Current Snapshot: MvccSnapshot[applied={T|T < 7315627352085483520}]
I20260806 18:43:54.023435 5882 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:54.023589 5882 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:54.026254 5882 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 324855646489441e9d736ba03c5ac49f: Sending DeleteTablet for 3 replicas of tablet a0630b0fbaf24dc6a7c1abd9a021348e
I20260806 18:43:54.027097 6072 tablet_service.cc:1514] Processing DeleteTablet for tablet a0630b0fbaf24dc6a7c1abd9a021348e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:54 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:54782
I20260806 18:43:54.027098 6479 tablet_service.cc:1514] Processing DeleteTablet for tablet a0630b0fbaf24dc6a7c1abd9a021348e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:54 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:36452
I20260806 18:43:54.027129 6271 tablet_service.cc:1514] Processing DeleteTablet for tablet a0630b0fbaf24dc6a7c1abd9a021348e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:54 UTC) from {username='kudu', principal='kudu/127.8.241.254@KRBTEST.COM'} at 127.0.0.1:52848
I20260806 18:43:54.027680 6627 tablet_replica.cc:330] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: stopping tablet replica
I20260806 18:43:54.027980 6630 tablet_replica.cc:330] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: stopping tablet replica
I20260806 18:43:54.028294 6630 raft_consensus.cc:2229] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:43:54.028473 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 5969
I20260806 18:43:54.028669 6630 raft_consensus.cc:2258] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:54.033226 6630 ts_tablet_manager.cc:1905] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:54.035329 6627 raft_consensus.cc:2229] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:54.035713 6627 raft_consensus.cc:2258] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:54.036511 6630 ts_tablet_manager.cc:1918] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:54.036680 6630 log.cc:1196] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/a0630b0fbaf24dc6a7c1abd9a021348e
I20260806 18:43:54.037122 6630 ts_tablet_manager.cc:1939] T a0630b0fbaf24dc6a7c1abd9a021348e P dc9455da4a0b44b89b7d3dd88d4ca312: Deleting consensus metadata
I20260806 18:43:54.038435 6627 ts_tablet_manager.cc:1905] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:54.038554 5859 catalog_manager.cc:4841] TS dc9455da4a0b44b89b7d3dd88d4ca312 (127.8.241.194:42703): tablet a0630b0fbaf24dc6a7c1abd9a021348e (table test-table [id=4490014e603b4d2482b0bb5f2570751c]) successfully deleted
I20260806 18:43:54.040700 6627 ts_tablet_manager.cc:1918] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:54.040795 6627 log.cc:1196] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/a0630b0fbaf24dc6a7c1abd9a021348e
I20260806 18:43:54.041087 6627 ts_tablet_manager.cc:1939] T a0630b0fbaf24dc6a7c1abd9a021348e P a37a49200c5c4381b9d1f819c9a66fb0: Deleting consensus metadata
I20260806 18:43:54.042201 5862 catalog_manager.cc:4841] TS a37a49200c5c4381b9d1f819c9a66fb0 (127.8.241.195:37031): tablet a0630b0fbaf24dc6a7c1abd9a021348e (table test-table [id=4490014e603b4d2482b0bb5f2570751c]) successfully deleted
W20260806 18:43:54.048550 5859 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:34119 (error 108)
I20260806 18:43:54.049475 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 6173
I20260806 18:43:54.049793 6595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.049441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:34119 (local address 127.0.0.1:54790)
0806 18:43:54.049585 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:54.049660 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:34119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 18:43:54.064110 5859 catalog_manager.cc:4584] TS e92daca9e65b4166b8a166c66027f967 (127.8.241.193:34119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0630b0fbaf24dc6a7c1abd9a021348e: Network error: Client connection negotiation failed: client connection to 127.8.241.193:34119: connect: Connection refused (error 111)
I20260806 18:43:54.070446 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 6380
I20260806 18:43:54.081872 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 5837
2026-08-06T18:43:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2579 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T18:43:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:43:54Z Disabled control of system clock
I20260806 18:43:54.117488 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34769
--webserver_interface=127.8.241.254
--webserver_port=0
--builtin_ntp_servers=127.8.241.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:34769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:43:54.229138 6639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:54.229409 6639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:54.229458 6639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:54.233379 6639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:54.233433 6639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:54.233453 6639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:54.233472 6639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:54.233493 6639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:54.238217 6639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:34769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:54.239297 6639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:54.240692 6639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:54.248914 6651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:54.248991 6655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:54.249099 6639 server_base.cc:1034] running on GCE node
W20260806 18:43:54.248934 6652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:54.249792 6639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:54.250890 6639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:54.252044 6639 hybrid_clock.cc:648] HybridClock initialized: now 1786041834252027 us; error 28 us; skew 500 ppm
I20260806 18:43:54.253834 6639 webserver.cc:460] Webserver started at http://127.8.241.254:44819/ using document root <none> and password file <none>
I20260806 18:43:54.254411 6639 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:54.254472 6639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:54.254678 6639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:54.256888 6639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "92678ac93e244441b783053ecf20ed59"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "72e7e8457a7b0e627755b50fe21e8aee"
server_key_iv: "b03c98359879e3c5ae45ab3dbe2f5c37"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.257511 6639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "92678ac93e244441b783053ecf20ed59"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "72e7e8457a7b0e627755b50fe21e8aee"
server_key_iv: "b03c98359879e3c5ae45ab3dbe2f5c37"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.261322 6639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:43:54.263756 6662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.264977 6639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:54.265120 6639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "92678ac93e244441b783053ecf20ed59"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "72e7e8457a7b0e627755b50fe21e8aee"
server_key_iv: "b03c98359879e3c5ae45ab3dbe2f5c37"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.265221 6639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:54.291944 6639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:54.292683 6639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:54.292882 6639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:54.299890 6639 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:34769
I20260806 18:43:54.299937 6737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:34769 every 8 connection(s)
I20260806 18:43:54.300869 6639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:54.303522 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 6639
I20260806 18:43:54.303623 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:43:54.303903 9159 external_mini_cluster.cc:1428] Setting key 58cdc26f505124485d7f9f25c834a0c4
I20260806 18:43:54.304236 6739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:54.311806 6739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59: Bootstrap starting.
I20260806 18:43:54.314607 6739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:54.314965 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.305396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.8.241.254:34769)
0806 18:43:54.306078 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:54.306098 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:54.306140 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:43:54.307086 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:54.307110 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:54.307640 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:54.308040 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:54.308502 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.309919 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.311164 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.311858 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.312015 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.312824 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:54.312831 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:54.312847 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:54.313381 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:54.313450 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:54.313477 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:54.313570 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:54.313665 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 18:43:54.313935 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":117,"threads_started":1}
I20260806 18:43:54.315510 6739 log.cc:824] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:54.317685 6739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59: No bootstrap required, opened a new log
I20260806 18:43:54.321298 6739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92678ac93e244441b783053ecf20ed59" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34769 } }
I20260806 18:43:54.321548 6739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:54.321595 6739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92678ac93e244441b783053ecf20ed59, State: Initialized, Role: FOLLOWER
I20260806 18:43:54.321981 6739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92678ac93e244441b783053ecf20ed59" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34769 } }
I20260806 18:43:54.322104 6739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:54.322149 6739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:54.322227 6739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:54.323040 6739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92678ac93e244441b783053ecf20ed59" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34769 } }
I20260806 18:43:54.323340 6739 leader_election.cc:302] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92678ac93e244441b783053ecf20ed59; no voters:
I20260806 18:43:54.323589 6739 leader_election.cc:288] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:43:54.323689 6746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:54.323949 6746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [term 1 LEADER]: Becoming Leader. State: Replica: 92678ac93e244441b783053ecf20ed59, State: Running, Role: LEADER
I20260806 18:43:54.324225 6746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92678ac93e244441b783053ecf20ed59" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34769 } }
I20260806 18:43:54.324728 6739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:54.325788 6750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92678ac93e244441b783053ecf20ed59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92678ac93e244441b783053ecf20ed59" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34769 } } }
I20260806 18:43:54.325927 6750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:54.326198 6753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92678ac93e244441b783053ecf20ed59. Latest consensus state: current_term: 1 leader_uuid: "92678ac93e244441b783053ecf20ed59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92678ac93e244441b783053ecf20ed59" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34769 } } }
I20260806 18:43:54.326284 6753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:54.326413 6767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:54.329628 6767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:54.335163 6767 catalog_manager.cc:1347] Generated new cluster ID: 94654246ace9481f8ff2e0b996b11d2b
I20260806 18:43:54.335227 6767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:54.346086 6767 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:43:54.347270 6767 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:54.357800 6767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59: Generated new TSK 0
I20260806 18:43:54.358588 6767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:43:54.367249 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.193:0
--local_ip_for_outbound_sockets=127.8.241.193
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34769
--builtin_ntp_servers=127.8.241.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:43:54.483438 6782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:54.483716 6782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:54.483757 6782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:54.487783 6782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:54.487864 6782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:54.487936 6782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.193
I20260806 18:43:54.492743 6782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.241.193
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.193
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:54.493908 6782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:54.495325 6782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:54.505086 6788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:54.505086 6789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:54.505249 6782 server_base.cc:1034] running on GCE node
W20260806 18:43:54.505116 6792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:54.505825 6782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:54.506506 6782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:54.507676 6782 hybrid_clock.cc:648] HybridClock initialized: now 1786041834507639 us; error 45 us; skew 500 ppm
I20260806 18:43:54.509691 6782 webserver.cc:460] Webserver started at http://127.8.241.193:42675/ using document root <none> and password file <none>
I20260806 18:43:54.510289 6782 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:54.510362 6782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:54.510571 6782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:54.512936 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/data/instance:
uuid: "bfda1edea97f446a91406b24b60b93d8"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "5c7fa6cc7ca3400f634ae49d481e36c1"
server_key_iv: "0f96fea8689a124abed4b68da8eb385f"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.513600 6782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance:
uuid: "bfda1edea97f446a91406b24b60b93d8"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "5c7fa6cc7ca3400f634ae49d481e36c1"
server_key_iv: "0f96fea8689a124abed4b68da8eb385f"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.517403 6782 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:43:54.519830 6801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.521075 6782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:54.521179 6782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/wal
uuid: "bfda1edea97f446a91406b24b60b93d8"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "5c7fa6cc7ca3400f634ae49d481e36c1"
server_key_iv: "0f96fea8689a124abed4b68da8eb385f"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.521276 6782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:54.538197 6782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:54.538923 6782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:54.539099 6782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:54.539700 6782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:54.540657 6782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:54.540704 6782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.540769 6782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:54.540797 6782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.549711 6782 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.193:41007
I20260806 18:43:54.549813 6968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.193:41007 every 8 connection(s)
I20260806 18:43:54.550648 6782 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/data/info.pb
I20260806 18:43:54.554169 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 6782
I20260806 18:43:54.554286 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-0/wal/instance
I20260806 18:43:54.554563 9159 external_mini_cluster.cc:1428] Setting key 76558ce656896a254960ceb762341ceb
I20260806 18:43:54.556773 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.194:0
--local_ip_for_outbound_sockets=127.8.241.194
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34769
--builtin_ntp_servers=127.8.241.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:43:54.560539 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.552474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.193:41959 (local address 127.8.241.254:34769)
0806 18:43:54.552643 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:54.552648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:54.553813 (+ 1165us) server_negotiation.cc:407] Connection header received
0806 18:43:54.555344 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:54.555347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:54.555395 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:54.555471 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:54.556577 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.557318 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.558589 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.558811 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.558886 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.560072 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:54.560075 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:54.560078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:54.560094 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:54.560118 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:54.560127 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:54.560199 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:54.560322 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 18:43:54.560397 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:43:54.561502 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.552941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34769 (local address 127.8.241.193:41959)
0806 18:43:54.553656 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:54.553697 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:54.555114 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:54.555673 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:54.555683 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:54.555790 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:54.556420 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.556436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.557448 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.557451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.558452 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.558466 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.558952 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.558957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.559163 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.559794 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:43:54.559796 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:54.559837 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:43:54.559917 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:54.560206 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:54.560211 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:54.560215 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:43:54.560306 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 18:43:54.560586 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":127,"threads_started":1}
I20260806 18:43:54.562764 6972 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34769
I20260806 18:43:54.563042 6972 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:54.563755 6972 heartbeater.cc:511] Master 127.8.241.254:34769 requested a full tablet report, sending...
I20260806 18:43:54.565667 6685 ts_manager.cc:194] Registered new tserver with Master: bfda1edea97f446a91406b24b60b93d8 (127.8.241.193:41007)
I20260806 18:43:54.567353 6685 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.193:41959
W20260806 18:43:54.675331 6978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:54.675583 6978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:54.675621 6978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:54.679584 6978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:54.679641 6978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:54.679693 6978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.194
I20260806 18:43:54.684326 6978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.241.194
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.194
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:54.685501 6978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:54.686884 6978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:54.696739 6986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:54.696724 6988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:54.696779 6978 server_base.cc:1034] running on GCE node
W20260806 18:43:54.696719 6985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:54.697398 6978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:54.698061 6978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:54.699326 6978 hybrid_clock.cc:648] HybridClock initialized: now 1786041834699296 us; error 37 us; skew 500 ppm
I20260806 18:43:54.701491 6978 webserver.cc:460] Webserver started at http://127.8.241.194:44793/ using document root <none> and password file <none>
I20260806 18:43:54.702092 6978 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:54.702162 6978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:54.702358 6978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:54.704456 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/data/instance:
uuid: "7c5aae2144b6465fae051c8df912a2e2"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "ee3d23eca429641f98b0b820bd25e7dd"
server_key_iv: "b3953c11deb9580b122805c2eeffd835"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.705046 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance:
uuid: "7c5aae2144b6465fae051c8df912a2e2"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "ee3d23eca429641f98b0b820bd25e7dd"
server_key_iv: "b3953c11deb9580b122805c2eeffd835"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.708878 6978 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 18:43:54.711436 6996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.712720 6978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:54.712870 6978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/wal
uuid: "7c5aae2144b6465fae051c8df912a2e2"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "ee3d23eca429641f98b0b820bd25e7dd"
server_key_iv: "b3953c11deb9580b122805c2eeffd835"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.712976 6978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:54.731314 6978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:54.732057 6978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:54.732236 6978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:54.732852 6978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:54.733875 6978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:54.733925 6978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.733980 6978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:54.734009 6978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.742481 6978 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.194:35571
I20260806 18:43:54.742524 7159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.194:35571 every 8 connection(s)
I20260806 18:43:54.743395 6978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/data/info.pb
I20260806 18:43:54.744395 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 6978
I20260806 18:43:54.744513 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/wal/instance
I20260806 18:43:54.744853 9159 external_mini_cluster.cc:1428] Setting key c41709c68e034e35b29a920a970fcdf7
I20260806 18:43:54.747052 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.241.195:0
--local_ip_for_outbound_sockets=127.8.241.195
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34769
--builtin_ntp_servers=127.8.241.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:43:54.753268 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.745336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.194:59645 (local address 127.8.241.254:34769)
0806 18:43:54.745473 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:54.745478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:54.746776 (+ 1298us) server_negotiation.cc:407] Connection header received
0806 18:43:54.748528 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:54.748531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:54.748577 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:54.748654 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:54.749614 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.750318 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.751419 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.751653 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.751741 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.752801 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:54.752805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:54.752807 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:54.752820 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:54.752841 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:54.752850 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:54.752912 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:54.753061 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 18:43:54.753125 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:43:54.754091 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.745836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34769 (local address 127.8.241.194:59645)
0806 18:43:54.746610 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:54.746665 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:54.748296 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:54.748844 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:54.748861 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:54.748946 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:54.749477 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.749492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.750441 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.750444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.751301 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.751310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.751799 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.751803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.751974 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.752560 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:43:54.752562 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:54.752601 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:43:54.752663 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:54.752922 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:54.752932 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:54.752938 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:43:54.753061 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 18:43:54.753329 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":150,"threads_started":1}
I20260806 18:43:54.755126 7160 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34769
I20260806 18:43:54.755345 7160 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:54.755848 7160 heartbeater.cc:511] Master 127.8.241.254:34769 requested a full tablet report, sending...
I20260806 18:43:54.756898 6685 ts_manager.cc:194] Registered new tserver with Master: 7c5aae2144b6465fae051c8df912a2e2 (127.8.241.194:35571)
I20260806 18:43:54.757684 6685 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.194:59645
W20260806 18:43:54.862061 7163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:54.862308 7163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:54.862346 7163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:54.866331 7163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:54.866389 7163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:54.866442 7163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.241.195
I20260806 18:43:54.871172 7163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.241.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.241.195
--webserver_port=0
--tserver_master_addrs=127.8.241.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.241.195
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:54.872200 7163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:54.873594 7163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:54.883172 7170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:54.883333 7163 server_base.cc:1034] running on GCE node
W20260806 18:43:54.883231 7169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:54.883235 7172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:54.883873 7163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:54.884629 7163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:54.885798 7163 hybrid_clock.cc:648] HybridClock initialized: now 1786041834885782 us; error 27 us; skew 500 ppm
I20260806 18:43:54.887635 7163 webserver.cc:460] Webserver started at http://127.8.241.195:33835/ using document root <none> and password file <none>
I20260806 18:43:54.888201 7163 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:54.888263 7163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:54.888463 7163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:54.890674 7163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/data/instance:
uuid: "cb0b7d2d0291412aa64a8506eda290a7"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "37fb7eac7dac0f24a6f7df047d14cadd"
server_key_iv: "58e937b055d799b58c905b571b566209"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.891218 7163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance:
uuid: "cb0b7d2d0291412aa64a8506eda290a7"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "37fb7eac7dac0f24a6f7df047d14cadd"
server_key_iv: "58e937b055d799b58c905b571b566209"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.894897 7163 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:43:54.897294 7185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.898444 7163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:43:54.898555 7163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/wal
uuid: "cb0b7d2d0291412aa64a8506eda290a7"
format_stamp: "Formatted at 2026-08-06 18:43:54 on dist-test-slave-k5rr"
server_key: "37fb7eac7dac0f24a6f7df047d14cadd"
server_key_iv: "58e937b055d799b58c905b571b566209"
server_key_version: "encryptionkey@0"
I20260806 18:43:54.898649 7163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:54.916815 7163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:54.917510 7163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:54.917670 7163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:54.918221 7163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:43:54.919145 7163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:43:54.919191 7163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.919229 7163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:43:54.919260 7163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:54.928121 7163 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.195:43207
I20260806 18:43:54.928159 7354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.195:43207 every 8 connection(s)
I20260806 18:43:54.929029 7163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/data/info.pb
I20260806 18:43:54.934693 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 7163
I20260806 18:43:54.934785 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/wal/instance
I20260806 18:43:54.935051 9159 external_mini_cluster.cc:1428] Setting key 1dd154865786250e8cddf52e573ee0f7
I20260806 18:43:54.938073 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.930819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:41327 (local address 127.8.241.254:34769)
0806 18:43:54.930960 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:54.930964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:54.932013 (+ 1049us) server_negotiation.cc:407] Connection header received
0806 18:43:54.933460 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:54.933463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:54.933534 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:54.933609 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:54.934522 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.935095 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.936329 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.936567 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.936648 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.937706 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:54.937710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:54.937712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:54.937727 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:54.937747 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:54.937756 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:54.937815 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:54.937913 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:43:54.937963 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:43:54.938898 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.931193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:34769 (local address 127.8.241.195:41327)
0806 18:43:54.931867 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:54.931907 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:54.933251 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:54.933795 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:54.933802 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:54.933885 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:43:54.934366 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.934379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.935226 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.935230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.936228 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.936238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.936709 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.936714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.936900 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.937471 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:43:54.937473 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:43:54.937513 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:43:54.937575 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:43:54.937820 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:43:54.937824 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:43:54.937828 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:43:54.937910 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:43:54.938126 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":121,"threads_started":1}
I20260806 18:43:54.939910 7355 heartbeater.cc:347] Connected to a master server at 127.8.241.254:34769
I20260806 18:43:54.940125 7355 heartbeater.cc:464] Registering TS with master...
I20260806 18:43:54.940603 7355 heartbeater.cc:511] Master 127.8.241.254:34769 requested a full tablet report, sending...
I20260806 18:43:54.941570 6685 ts_manager.cc:194] Registered new tserver with Master: cb0b7d2d0291412aa64a8506eda290a7 (127.8.241.195:43207)
I20260806 18:43:54.942108 6685 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.241.195:41327
I20260806 18:43:54.948967 9159 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:43:54.954128 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.950872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55830 (local address 127.8.241.254:34769)
0806 18:43:54.951000 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:54.951004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:54.951159 (+ 155us) server_negotiation.cc:407] Connection header received
0806 18:43:54.951256 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:54.951258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:54.951299 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:54.951359 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:54.951592 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.952082 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.952870 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.953085 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.953157 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.953791 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:54.953794 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:54.953796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:54.953807 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:54.953824 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:54.953831 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:54.953885 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:54.953943 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 18:43:54.953988 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:43:54.956756 6685 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:43:54.959851 6685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:43:54.974068 7370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.968092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:35571 (local address 127.0.0.1:59424)
0806 18:43:54.968891 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:54.968942 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:54.969092 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:54.969947 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:54.969953 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:54.969977 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:54.970113 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.970123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.971615 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.971618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.972624 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.972635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.973677 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.973680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.973762 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.973787 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:43:54.973832 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 18:43:54.973921 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":122,"thread_start_us":80,"threads_started":1}
I20260806 18:43:54.974083 7374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.969034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.8.241.195:43207)
0806 18:43:54.969429 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:54.969443 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:54.969696 (+ 253us) server_negotiation.cc:407] Connection header received
0806 18:43:54.969868 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:54.969872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:54.970066 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:54.970213 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:54.970447 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.971182 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.972739 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.973333 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.973445 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.973657 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:54.973784 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 18:43:54.973890 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":196,"threads_started":1}
I20260806 18:43:54.974401 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.968298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59424 (local address 127.8.241.194:35571)
0806 18:43:54.969444 (+ 1146us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:54.969457 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:54.969484 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:43:54.969573 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:54.969580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:54.969793 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:54.969961 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:54.970246 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.971481 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.972765 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.973557 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.973914 (+ 357us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.974087 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:54.974134 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 18:43:54.974216 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":126,"threads_started":1}
I20260806 18:43:54.974984 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.968209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:41007 (local address 127.0.0.1:34362)
0806 18:43:54.969314 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:54.969326 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:54.969420 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:54.970137 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:54.970140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:54.970159 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:54.970282 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.970290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.971616 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.971618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.973529 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.973541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.974605 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.974609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.974693 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.974714 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:43:54.974790 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:43:54.974856 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":43,"thread_start_us":90,"threads_started":1}
I20260806 18:43:54.975131 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.969291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.8.241.193:41007)
0806 18:43:54.969651 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:54.969663 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:54.969685 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:43:54.969774 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:54.969779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:54.969948 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:54.970086 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:54.970397 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.971509 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.973672 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.974417 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.974515 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.974689 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:54.974855 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 18:43:54.974961 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":129,"threads_started":1}
I20260806 18:43:54.975641 7371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:54.968812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.195:43207 (local address 127.0.0.1:51890)
0806 18:43:54.969593 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:54.969605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:54.969695 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:54.970236 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:54.970239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:54.970256 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:54.970343 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.970350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.971341 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.971345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.972602 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:54.972613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:54.973459 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:54.973463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:54.973583 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:54.973626 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 18:43:54.973753 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 18:43:54.975524 (+ 1771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":54,"thread_start_us":99,"threads_started":1}
I20260806 18:43:54.977609 7071 tablet_service.cc:1467] Processing CreateTablet for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d (DEFAULT_TABLE table=test-table [id=ee9da98078db42659af8649e597fcc2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:54.978029 6872 tablet_service.cc:1467] Processing CreateTablet for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d (DEFAULT_TABLE table=test-table [id=ee9da98078db42659af8649e597fcc2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:54.978211 7262 tablet_service.cc:1467] Processing CreateTablet for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d (DEFAULT_TABLE table=test-table [id=ee9da98078db42659af8649e597fcc2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:43:54.978662 7071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:54.979087 7262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:54.979087 6872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:54.985298 7378 tablet_bootstrap.cc:492] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: Bootstrap starting.
I20260806 18:43:54.986801 7379 tablet_bootstrap.cc:492] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: Bootstrap starting.
I20260806 18:43:54.988659 7378 tablet_bootstrap.cc:654] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:54.989104 7379 tablet_bootstrap.cc:654] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:54.989349 7380 tablet_bootstrap.cc:492] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8: Bootstrap starting.
I20260806 18:43:54.989735 7378 log.cc:824] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:54.989969 7379 log.cc:824] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:54.991315 7380 tablet_bootstrap.cc:654] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:54.992089 7379 tablet_bootstrap.cc:492] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: No bootstrap required, opened a new log
I20260806 18:43:54.992066 7380 log.cc:824] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:54.992609 7378 tablet_bootstrap.cc:492] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: No bootstrap required, opened a new log
I20260806 18:43:54.992714 7379 ts_tablet_manager.cc:1397] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 18:43:54.992794 7378 ts_tablet_manager.cc:1397] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 18:43:54.993813 7380 tablet_bootstrap.cc:492] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8: No bootstrap required, opened a new log
I20260806 18:43:54.993961 7380 ts_tablet_manager.cc:1397] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:43:54.996987 7380 raft_consensus.cc:348] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } }
I20260806 18:43:54.997078 7378 raft_consensus.cc:348] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } }
I20260806 18:43:54.997078 7379 raft_consensus.cc:348] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } }
I20260806 18:43:54.997318 7380 raft_consensus.cc:374] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:54.997345 7378 raft_consensus.cc:374] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:54.997345 7379 raft_consensus.cc:374] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:54.997393 7380 raft_consensus.cc:729] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfda1edea97f446a91406b24b60b93d8, State: Initialized, Role: FOLLOWER
I20260806 18:43:54.997411 7379 raft_consensus.cc:729] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0b7d2d0291412aa64a8506eda290a7, State: Initialized, Role: FOLLOWER
I20260806 18:43:54.997468 7378 raft_consensus.cc:729] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c5aae2144b6465fae051c8df912a2e2, State: Initialized, Role: FOLLOWER
I20260806 18:43:54.997836 7380 consensus_queue.cc:260] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } }
I20260806 18:43:54.997951 7378 consensus_queue.cc:260] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } }
I20260806 18:43:54.997951 7379 consensus_queue.cc:260] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } }
I20260806 18:43:54.998713 7355 heartbeater.cc:503] Master 127.8.241.254:34769 was elected leader, sending a full tablet report...
I20260806 18:43:54.999003 7378 ts_tablet_manager.cc:1428] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:43:54.999003 7379 ts_tablet_manager.cc:1428] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260806 18:43:54.999575 7160 heartbeater.cc:503] Master 127.8.241.254:34769 was elected leader, sending a full tablet report...
I20260806 18:43:55.000800 7380 ts_tablet_manager.cc:1428] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260806 18:43:55.001176 6972 heartbeater.cc:503] Master 127.8.241.254:34769 was elected leader, sending a full tablet report...
W20260806 18:43:55.051847 6973 tablet.cc:2369] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:55.168686 7388 raft_consensus.cc:482] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:43:55.168913 7388 raft_consensus.cc:504] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } }
I20260806 18:43:55.170238 7388 leader_election.cc:288] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c5aae2144b6465fae051c8df912a2e2 (127.8.241.194:35571), bfda1edea97f446a91406b24b60b93d8 (127.8.241.193:41007)
I20260806 18:43:55.173866 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:55.170579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:59501 (local address 127.8.241.194:35571)
0806 18:43:55.170715 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:55.170759 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:55.170776 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:43:55.170840 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:55.170843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:55.170895 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:55.170981 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:55.171261 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.171796 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.172912 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.173355 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.173464 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:55.173519 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:55.173685 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 18:43:55.173766 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:43:55.173866 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:55.170454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.194:35571 (local address 127.8.241.195:59501)
0806 18:43:55.170585 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:55.170601 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:55.170700 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:55.170994 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:55.170998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:55.171039 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:55.171144 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:55.171150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.171909 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.171913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:55.172782 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:55.172792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.173465 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.173469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:55.173571 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:55.173589 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:43:55.173656 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:43:55.173726 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":26,"spinlock_wait_cycles":24704}
I20260806 18:43:55.173957 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:55.170756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.241.195:43107 (local address 127.8.241.193:41007)
0806 18:43:55.170906 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:55.170910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:55.171022 (+ 112us) server_negotiation.cc:407] Connection header received
0806 18:43:55.171153 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:55.171155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:55.171201 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:55.171262 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:43:55.171549 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.172028 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.173060 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.173428 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.173531 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:55.173583 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:55.173758 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 18:43:55.173827 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:43:55.173992 7391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:55.170520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:41007 (local address 127.8.241.195:43107)
0806 18:43:55.170933 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:55.170943 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:43:55.171053 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:43:55.171289 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:43:55.171291 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:43:55.171307 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:43:55.171432 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:55.171438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.172133 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.172136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:55.172929 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:43:55.172938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.173556 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.173559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:43:55.173648 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:55.173666 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:43:55.173730 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:43:55.173796 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":1,"spinlock_wait_cycles":384,"thread_start_us":133,"threads_started":1}
I20260806 18:43:55.174589 7098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1005db44fe46f4b6bdc7b965cb8b6d" candidate_uuid: "cb0b7d2d0291412aa64a8506eda290a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c5aae2144b6465fae051c8df912a2e2" is_pre_election: true
I20260806 18:43:55.174793 6903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1005db44fe46f4b6bdc7b965cb8b6d" candidate_uuid: "cb0b7d2d0291412aa64a8506eda290a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfda1edea97f446a91406b24b60b93d8" is_pre_election: true
I20260806 18:43:55.174896 7098 raft_consensus.cc:2454] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0b7d2d0291412aa64a8506eda290a7 in term 0.
I20260806 18:43:55.175076 6903 raft_consensus.cc:2454] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0b7d2d0291412aa64a8506eda290a7 in term 0.
I20260806 18:43:55.175417 7188 leader_election.cc:302] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c5aae2144b6465fae051c8df912a2e2, cb0b7d2d0291412aa64a8506eda290a7; no voters:
I20260806 18:43:55.175675 7388 raft_consensus.cc:2781] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:43:55.175767 7388 raft_consensus.cc:482] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:43:55.175802 7388 raft_consensus.cc:3037] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:55.176849 7388 raft_consensus.cc:504] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } }
I20260806 18:43:55.177232 7388 leader_election.cc:288] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c5aae2144b6465fae051c8df912a2e2 (127.8.241.194:35571), bfda1edea97f446a91406b24b60b93d8 (127.8.241.193:41007)
I20260806 18:43:55.177536 7098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1005db44fe46f4b6bdc7b965cb8b6d" candidate_uuid: "cb0b7d2d0291412aa64a8506eda290a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c5aae2144b6465fae051c8df912a2e2"
I20260806 18:43:55.177557 6903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f1005db44fe46f4b6bdc7b965cb8b6d" candidate_uuid: "cb0b7d2d0291412aa64a8506eda290a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfda1edea97f446a91406b24b60b93d8"
I20260806 18:43:55.177666 6903 raft_consensus.cc:3037] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:55.177666 7098 raft_consensus.cc:3037] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:55.178930 7098 raft_consensus.cc:2454] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0b7d2d0291412aa64a8506eda290a7 in term 1.
I20260806 18:43:55.178930 6903 raft_consensus.cc:2454] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0b7d2d0291412aa64a8506eda290a7 in term 1.
I20260806 18:43:55.179242 7187 leader_election.cc:302] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfda1edea97f446a91406b24b60b93d8, cb0b7d2d0291412aa64a8506eda290a7; no voters:
I20260806 18:43:55.179409 7388 raft_consensus.cc:2781] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:55.179651 7388 raft_consensus.cc:686] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 1 LEADER]: Becoming Leader. State: Replica: cb0b7d2d0291412aa64a8506eda290a7, State: Running, Role: LEADER
W20260806 18:43:55.179903 7357 tablet.cc:2369] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:43:55.179999 7388 consensus_queue.cc:237] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } }
I20260806 18:43:55.183255 6684 catalog_manager.cc:5495] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb0b7d2d0291412aa64a8506eda290a7 (127.8.241.195). New cstate: current_term: 1 leader_uuid: "cb0b7d2d0291412aa64a8506eda290a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb0b7d2d0291412aa64a8506eda290a7" member_type: VOTER last_known_addr { host: "127.8.241.195" port: 43207 } health_report { overall_health: HEALTHY } } }
I20260806 18:43:55.212013 7374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:55.208577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51898 (local address 127.8.241.195:43207)
0806 18:43:55.208719 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:55.208725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:55.208743 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:43:55.208824 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:55.208827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:55.208891 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:55.208985 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:43:55.209231 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.209749 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.210576 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:55.210797 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:55.210885 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:55.211227 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:43:55.211669 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:43:55.211784 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:55.211841 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 18:43:55.211894 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:43:55.218091 7098 raft_consensus.cc:1261] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [term 1 FOLLOWER]: Refusing update from remote peer cb0b7d2d0291412aa64a8506eda290a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:55.218091 6903 raft_consensus.cc:1261] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8 [term 1 FOLLOWER]: Refusing update from remote peer cb0b7d2d0291412aa64a8506eda290a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:43:55.218858 7392 consensus_queue.cc:1035] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfda1edea97f446a91406b24b60b93d8" member_type: VOTER last_known_addr { host: "127.8.241.193" port: 41007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:55.219049 7388 consensus_queue.cc:1035] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c5aae2144b6465fae051c8df912a2e2" member_type: VOTER last_known_addr { host: "127.8.241.194" port: 35571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:43:55.227480 7403 mvcc.cc:205] Tried to move back new op lower bound from 7315627357045481472 to 7315627356899876864. Current Snapshot: MvccSnapshot[applied={T|T < 7315627357045481472}]
I20260806 18:43:55.236560 6684 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:55.236722 6684 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:43:55.239154 6684 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92678ac93e244441b783053ecf20ed59: Sending DeleteTablet for 3 replicas of tablet 6f1005db44fe46f4b6bdc7b965cb8b6d
I20260806 18:43:55.239939 7071 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:55 UTC) from {username='slave'} at 127.0.0.1:59424
I20260806 18:43:55.239995 7262 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:55 UTC) from {username='slave'} at 127.0.0.1:51890
I20260806 18:43:55.240623 6872 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:43:55 UTC) from {username='slave'} at 127.0.0.1:34362
I20260806 18:43:55.240754 7417 tablet_replica.cc:330] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: stopping tablet replica
I20260806 18:43:55.241099 7418 tablet_replica.cc:330] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8: stopping tablet replica
I20260806 18:43:55.241091 7417 raft_consensus.cc:2229] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:43:55.241189 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 6782
I20260806 18:43:55.241432 7418 raft_consensus.cc:2229] T 6f1005db44fe46f4b6bdc7b965cb8b6d P bfda1edea97f446a91406b24b60b93d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:55.241446 7417 raft_consensus.cc:2258] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:55.241843 7416 tablet_replica.cc:330] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: stopping tablet replica
I20260806 18:43:55.242192 7416 raft_consensus.cc:2229] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:43:55.242540 7416 raft_consensus.cc:2258] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:43:55.248144 7417 ts_tablet_manager.cc:1905] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:55.250923 7416 ts_tablet_manager.cc:1905] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:43:55.251274 7417 ts_tablet_manager.cc:1918] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:55.251436 7417 log.cc:1196] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-2/wal/wals/6f1005db44fe46f4b6bdc7b965cb8b6d
I20260806 18:43:55.251847 7417 ts_tablet_manager.cc:1939] T 6f1005db44fe46f4b6bdc7b965cb8b6d P cb0b7d2d0291412aa64a8506eda290a7: Deleting consensus metadata
I20260806 18:43:55.253368 6663 catalog_manager.cc:4841] TS cb0b7d2d0291412aa64a8506eda290a7 (127.8.241.195:43207): tablet 6f1005db44fe46f4b6bdc7b965cb8b6d (table test-table [id=ee9da98078db42659af8649e597fcc2e]) successfully deleted
I20260806 18:43:55.254376 7416 ts_tablet_manager.cc:1918] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:43:55.254485 7416 log.cc:1196] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: Deleting WAL directory at /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041577757111-9159-0/minicluster-data/ts-1/wal/wals/6f1005db44fe46f4b6bdc7b965cb8b6d
I20260806 18:43:55.254840 7416 ts_tablet_manager.cc:1939] T 6f1005db44fe46f4b6bdc7b965cb8b6d P 7c5aae2144b6465fae051c8df912a2e2: Deleting consensus metadata
I20260806 18:43:55.256196 6664 catalog_manager.cc:4841] TS 7c5aae2144b6465fae051c8df912a2e2 (127.8.241.194:35571): tablet 6f1005db44fe46f4b6bdc7b965cb8b6d (table test-table [id=ee9da98078db42659af8649e597fcc2e]) successfully deleted
I20260806 18:43:55.257762 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 6978
W20260806 18:43:55.257777 6663 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.241.193:41007 (error 108)
I20260806 18:43:55.267709 7371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:55.267345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.193:41007 (local address 127.0.0.1:34368)
0806 18:43:55.267518 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 18:43:55.267602 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.193:41007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 18:43:55.267946 6663 catalog_manager.cc:4584] TS bfda1edea97f446a91406b24b60b93d8 (127.8.241.193:41007): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f1005db44fe46f4b6bdc7b965cb8b6d: Network error: Client connection negotiation failed: client connection to 127.8.241.193:41007: connect: Connection refused (error 111)
I20260806 18:43:55.275424 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 7163
I20260806 18:43:55.291152 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 6639
2026-08-06T18:43:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1210 ms)
[----------] 6 tests from EncryptionPolicyTest (10820 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:43:55 dist-test-slave-k5rr krb5kdc[7424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:43:55 dist-test-slave-k5rr krb5kdc[7424](info): set up 2 sockets
Aug 06 18:43:55 dist-test-slave-k5rr krb5kdc[7424](info): commencing operation
krb5kdc: starting...
W20260806 18:43:56.420423 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:43:56 dist-test-slave-k5rr krb5kdc[7424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:43:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:43:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:43:56.543462 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39389
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:56.656522 7441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:56.656787 7441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:56.656845 7441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:56.660835 7441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:56.660898 7441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:56.660925 7441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:56.660954 7441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:56.660983 7441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:56.666206 7441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39389
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:56.667263 7441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:56.668270 7441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:56.676741 7449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:56.676783 7441 server_base.cc:1034] running on GCE node
W20260806 18:43:56.676780 7451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:56.676923 7453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:56.677742 7441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:56.678891 7441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:56.680063 7441 hybrid_clock.cc:648] HybridClock initialized: now 1786041836680044 us; error 31 us; skew 500 ppm
Aug 06 18:43:56 dist-test-slave-k5rr krb5kdc[7424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041836, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:56.682767 7441 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:43:56.683671 7441 webserver.cc:460] Webserver started at http://127.0.0.1:41295/ using document root <none> and password file <none>
I20260806 18:43:56.684230 7441 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:56.684298 7441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:56.684521 7441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:56.686652 7441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "c3cc936a1de646ae932ea5ac3bc13415"
format_stamp: "Formatted at 2026-08-06 18:43:56 on dist-test-slave-k5rr"
server_key: "4b20fa0d4565b57ac80e6ad9696413e0"
server_key_iv: "eec61d096d1b58b2268c2f47d94f9bb7"
server_key_version: "encryptionkey@0"
I20260806 18:43:56.687199 7441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "c3cc936a1de646ae932ea5ac3bc13415"
format_stamp: "Formatted at 2026-08-06 18:43:56 on dist-test-slave-k5rr"
server_key: "4b20fa0d4565b57ac80e6ad9696413e0"
server_key_iv: "eec61d096d1b58b2268c2f47d94f9bb7"
server_key_version: "encryptionkey@0"
I20260806 18:43:56.691056 7441 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:43:56.693550 7461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:56.694773 7441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:43:56.694918 7441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "c3cc936a1de646ae932ea5ac3bc13415"
format_stamp: "Formatted at 2026-08-06 18:43:56 on dist-test-slave-k5rr"
server_key: "4b20fa0d4565b57ac80e6ad9696413e0"
server_key_iv: "eec61d096d1b58b2268c2f47d94f9bb7"
server_key_version: "encryptionkey@0"
I20260806 18:43:56.695020 7441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:56.724843 7441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:56.727720 7441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:56.727921 7441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:56.734951 7441 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39389
I20260806 18:43:56.734977 7534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39389 every 8 connection(s)
I20260806 18:43:56.735900 7441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:56.738766 7536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:56.739611 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 7441
I20260806 18:43:56.739714 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:43:56.739989 9159 external_mini_cluster.cc:1428] Setting key 610ad0276f4f9f50e22440f3434e39ca
I20260806 18:43:56.745631 7536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415: Bootstrap starting.
I20260806 18:43:56.748186 7536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:56.748914 7536 log.cc:824] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:43:56 dist-test-slave-k5rr krb5kdc[7424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041836, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:43:56.751223 7536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415: No bootstrap required, opened a new log
I20260806 18:43:56.754870 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:56.741417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44741 (local address 127.0.0.1:39389)
0806 18:43:56.741842 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:56.741860 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:56.741905 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:43:56.742750 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:56.742776 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:56.743209 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:56.743608 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:56.744276 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:56.745791 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:56.746606 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:56.747283 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:56.747437 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:56.749836 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:56.749842 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:56.749859 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:56.749889 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:56.751727 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:56.752200 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:56.752205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:56.752211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:56.752356 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:56.752617 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:56.752620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:56.752622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:56.753168 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:56.753391 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:56.753628 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 18:43:56.753875 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":98,"threads_started":1}
I20260806 18:43:56.755767 7536 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cc936a1de646ae932ea5ac3bc13415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39389 } }
I20260806 18:43:56.756003 7536 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:56.756052 7536 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3cc936a1de646ae932ea5ac3bc13415, State: Initialized, Role: FOLLOWER
I20260806 18:43:56.756500 7536 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cc936a1de646ae932ea5ac3bc13415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39389 } }
I20260806 18:43:56.756651 7536 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:56.756718 7536 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:56.756841 7536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:56.757809 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:43:56.757798 7536 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cc936a1de646ae932ea5ac3bc13415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39389 } }
I20260806 18:43:56.758181 7536 leader_election.cc:302] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3cc936a1de646ae932ea5ac3bc13415; no voters:
I20260806 18:43:56.758529 7536 leader_election.cc:288] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:43:56.758616 7541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:56.758821 7541 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [term 1 LEADER]: Becoming Leader. State: Replica: c3cc936a1de646ae932ea5ac3bc13415, State: Running, Role: LEADER
I20260806 18:43:56.759174 7541 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cc936a1de646ae932ea5ac3bc13415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39389 } }
I20260806 18:43:56.759892 7536 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:56.760725 7543 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3cc936a1de646ae932ea5ac3bc13415. Latest consensus state: current_term: 1 leader_uuid: "c3cc936a1de646ae932ea5ac3bc13415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cc936a1de646ae932ea5ac3bc13415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39389 } } }
I20260806 18:43:56.760878 7543 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:56.761224 7542 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3cc936a1de646ae932ea5ac3bc13415" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3cc936a1de646ae932ea5ac3bc13415" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39389 } } }
I20260806 18:43:56.761322 7542 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:56.761615 7561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:56.764819 7561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:56.770784 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:56.760578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46155 (local address 127.0.0.1:39389)
0806 18:43:56.760997 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:56.761095 (+ 98us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:56.761951 (+ 856us) server_negotiation.cc:407] Connection header received
0806 18:43:56.762265 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:56.762268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:56.762314 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:56.762386 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:56.763129 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:56.763925 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:56.765189 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:56.765478 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:56.765538 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:56.767122 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:56.767126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:56.767129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:43:56.767154 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:56.768734 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:56.769513 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:56.769517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:56.769519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:56.769614 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:43:56.769923 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:43:56.769927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:43:56.769928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:43:56.770150 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:56.770296 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:56.770520 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 18:43:56.770626 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342}
I20260806 18:43:56.774402 7561 catalog_manager.cc:1347] Generated new cluster ID: 675426a0e48d4e0f9a11e5ef83c84483
I20260806 18:43:56.774479 7561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:56.790056 7561 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:43:56.790971 7561 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:56.801359 7561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3cc936a1de646ae932ea5ac3bc13415: Generated new TSK 0
I20260806 18:43:56.802053 7561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:43:56.807276 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 7441
W20260806 18:43:56.823616 7421 connection.cc:535] client connection to 127.0.0.1:39389 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39389): Connection reset by peer (error 104)
2026-08-06T18:43:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:43:56 dist-test-slave-k5rr krb5kdc[7582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:43:56 dist-test-slave-k5rr krb5kdc[7582](info): set up 2 sockets
Aug 06 18:43:56 dist-test-slave-k5rr krb5kdc[7582](info): commencing operation
krb5kdc: starting...
W20260806 18:43:57.941737 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:43:58 dist-test-slave-k5rr krb5kdc[7582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:43:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:43:58.063870 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34651
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:58.175201 7599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:58.175488 7599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:58.175530 7599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:58.179517 7599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:58.179576 7599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:58.179593 7599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:58.179608 7599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:58.179622 7599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:58.184818 7599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34651
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:58.185899 7599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:58.186932 7599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:58.195439 7606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:58.195442 7607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:58.195518 7609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:58.195652 7599 server_base.cc:1034] running on GCE node
I20260806 18:43:58.196291 7599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:58.197507 7599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:58.198688 7599 hybrid_clock.cc:648] HybridClock initialized: now 1786041838198659 us; error 38 us; skew 500 ppm
Aug 06 18:43:58 dist-test-slave-k5rr krb5kdc[7582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041838, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:58.204468 7599 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:43:58.205506 7599 webserver.cc:460] Webserver started at http://127.0.0.1:38833/ using document root <none> and password file <none>
I20260806 18:43:58.206081 7599 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:58.206148 7599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:58.206357 7599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:58.208446 7599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "01f06f2d9c0a42efa7d265c669622f77"
format_stamp: "Formatted at 2026-08-06 18:43:58 on dist-test-slave-k5rr"
server_key: "62c4b1eaa65446b8617fd54646eb609a"
server_key_iv: "9202987f211f8c0a4a6459de88e5e531"
server_key_version: "encryptionkey@0"
I20260806 18:43:58.209038 7599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "01f06f2d9c0a42efa7d265c669622f77"
format_stamp: "Formatted at 2026-08-06 18:43:58 on dist-test-slave-k5rr"
server_key: "62c4b1eaa65446b8617fd54646eb609a"
server_key_iv: "9202987f211f8c0a4a6459de88e5e531"
server_key_version: "encryptionkey@0"
I20260806 18:43:58.212863 7599 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:43:58.215312 7618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:58.216499 7599 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:58.216619 7599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "01f06f2d9c0a42efa7d265c669622f77"
format_stamp: "Formatted at 2026-08-06 18:43:58 on dist-test-slave-k5rr"
server_key: "62c4b1eaa65446b8617fd54646eb609a"
server_key_iv: "9202987f211f8c0a4a6459de88e5e531"
server_key_version: "encryptionkey@0"
I20260806 18:43:58.216710 7599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:58.228775 7599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:58.231905 7599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:58.232084 7599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:58.239027 7599 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34651
I20260806 18:43:58.239040 7692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34651 every 8 connection(s)
I20260806 18:43:58.239971 7599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:58.242772 7693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:58.248535 7693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77: Bootstrap starting.
I20260806 18:43:58.250082 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 7599
I20260806 18:43:58.250213 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:43:58.250499 9159 external_mini_cluster.cc:1428] Setting key 48ee9bc08c7e6c924b55ff6c6cc14ab0
I20260806 18:43:58.251276 7693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:58.252188 7693 log.cc:824] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:58.254496 7693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77: No bootstrap required, opened a new log
I20260806 18:43:58.258626 7693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f06f2d9c0a42efa7d265c669622f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34651 } }
I20260806 18:43:58.258899 7693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:58.258962 7693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01f06f2d9c0a42efa7d265c669622f77, State: Initialized, Role: FOLLOWER
I20260806 18:43:58.259467 7693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f06f2d9c0a42efa7d265c669622f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34651 } }
I20260806 18:43:58.259639 7693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:58.259734 7693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:58.259853 7693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:58.260406 7702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:58.251660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60773 (local address 127.0.0.1:34651)
0806 18:43:58.252154 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:58.252173 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:58.252219 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:43:58.252983 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:58.253031 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:58.253454 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:58.253817 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:58.254258 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:58.255675 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:58.256605 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:58.257409 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:58.257572 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:58.258354 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:58.258360 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:58.258376 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:58.258873 (+ 497us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:58.258945 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:58.258979 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:58.259108 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:58.259261 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 18:43:58.259392 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":132,"threads_started":1}
I20260806 18:43:58.260946 7693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f06f2d9c0a42efa7d265c669622f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34651 } }
I20260806 18:43:58.261291 7693 leader_election.cc:302] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01f06f2d9c0a42efa7d265c669622f77; no voters:
I20260806 18:43:58.261606 7693 leader_election.cc:288] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:43:58.261683 7704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:58.261883 7704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [term 1 LEADER]: Becoming Leader. State: Replica: 01f06f2d9c0a42efa7d265c669622f77, State: Running, Role: LEADER
I20260806 18:43:58.262254 7704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f06f2d9c0a42efa7d265c669622f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34651 } }
I20260806 18:43:58.262920 7693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:58.263547 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:43:58.263898 7705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01f06f2d9c0a42efa7d265c669622f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f06f2d9c0a42efa7d265c669622f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34651 } } }
I20260806 18:43:58.264017 7705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:58.264219 7705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01f06f2d9c0a42efa7d265c669622f77. Latest consensus state: current_term: 1 leader_uuid: "01f06f2d9c0a42efa7d265c669622f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01f06f2d9c0a42efa7d265c669622f77" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34651 } } }
I20260806 18:43:58.264290 7705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:58.264572 7715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:58.267796 7715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:58.272141 7702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:58.265910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44031 (local address 127.0.0.1:34651)
0806 18:43:58.266017 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:58.266022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:58.266307 (+ 285us) server_negotiation.cc:407] Connection header received
0806 18:43:58.266450 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:58.266453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:58.266513 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:58.266592 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:58.266872 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:58.267626 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:58.268699 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:43:58.269084 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:43:58.269232 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:43:58.271615 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:58.271618 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:58.271621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:58.271635 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:58.271661 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:58.271671 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:58.271740 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:58.271967 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 18:43:58.272007 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 18:43:58.274056 7715 catalog_manager.cc:1347] Generated new cluster ID: f678132636d949cd8a1420dfb9a95a1b
I20260806 18:43:58.274121 7715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:58.288197 7715 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:43:58.289181 7715 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:58.294140 7715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01f06f2d9c0a42efa7d265c669622f77: Generated new TSK 0
I20260806 18:43:58.294852 7715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:43:58.299075 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 7599
W20260806 18:43:58.314515 7579 connection.cc:535] client connection to 127.0.0.1:34651 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34651): Connection reset by peer (error 104)
2026-08-06T18:43:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:43:58 dist-test-slave-k5rr krb5kdc[7738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:43:58 dist-test-slave-k5rr krb5kdc[7738](info): set up 2 sockets
Aug 06 18:43:58 dist-test-slave-k5rr krb5kdc[7738](info): commencing operation
krb5kdc: starting...
W20260806 18:43:59.428653 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:43:59 dist-test-slave-k5rr krb5kdc[7738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041839, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:43:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:43:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:43:59.551890 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:43:59.663698 7758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:43:59.663967 7758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:43:59.664008 7758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:43:59.667910 7758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:43:59.667973 7758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:43:59.667991 7758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:43:59.668006 7758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:43:59.668021 7758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:43:59.673350 7758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37511
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:43:59.674448 7758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:43:59.675484 7758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:43:59.684054 7769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:59.684252 7758 server_base.cc:1034] running on GCE node
W20260806 18:43:59.684074 7766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:43:59.684053 7767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:43:59.684928 7758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:43:59.686065 7758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:43:59.687213 7758 hybrid_clock.cc:648] HybridClock initialized: now 1786041839687189 us; error 39 us; skew 500 ppm
Aug 06 18:43:59 dist-test-slave-k5rr krb5kdc[7738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041839, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:43:59.690001 7758 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:43:59.690900 7758 webserver.cc:460] Webserver started at http://127.0.0.1:44179/ using document root <none> and password file <none>
I20260806 18:43:59.691468 7758 fs_manager.cc:359] Metadata directory not provided
I20260806 18:43:59.691536 7758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:43:59.691747 7758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:43:59.693861 7758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "42adac2ca9d9422db50436030be3b1e9"
format_stamp: "Formatted at 2026-08-06 18:43:59 on dist-test-slave-k5rr"
server_key: "38711b69b6215d405aaff9ae7beeef3b"
server_key_iv: "a6da8399c4ef5de550e0677201dac25d"
server_key_version: "encryptionkey@0"
I20260806 18:43:59.694403 7758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "42adac2ca9d9422db50436030be3b1e9"
format_stamp: "Formatted at 2026-08-06 18:43:59 on dist-test-slave-k5rr"
server_key: "38711b69b6215d405aaff9ae7beeef3b"
server_key_iv: "a6da8399c4ef5de550e0677201dac25d"
server_key_version: "encryptionkey@0"
I20260806 18:43:59.698230 7758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260806 18:43:59.700604 7785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:43:59.701818 7758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:43:59.701965 7758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "42adac2ca9d9422db50436030be3b1e9"
format_stamp: "Formatted at 2026-08-06 18:43:59 on dist-test-slave-k5rr"
server_key: "38711b69b6215d405aaff9ae7beeef3b"
server_key_iv: "a6da8399c4ef5de550e0677201dac25d"
server_key_version: "encryptionkey@0"
I20260806 18:43:59.702067 7758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:43:59.706565 7758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:43:59.709491 7758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:43:59.709654 7758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:43:59.716513 7758 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37511
I20260806 18:43:59.716531 7859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37511 every 8 connection(s)
I20260806 18:43:59.717460 7758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:43:59.718062 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 7758
I20260806 18:43:59.718245 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:43:59.718555 9159 external_mini_cluster.cc:1428] Setting key 125b31439c0b776a7085d38451c4c511
I20260806 18:43:59.720628 7861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:43:59.724030 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:59.719925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50459 (local address 127.0.0.1:37511)
0806 18:43:59.720351 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:59.720371 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:59.720413 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:43:59.721311 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:59.721334 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:59.721678 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:59.722038 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:59.722689 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:59.722695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:59.722710 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:59.722720 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:59.722776 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:59.722800 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:59.722943 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:59.723008 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 18:43:59.723153 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260806 18:43:59.726915 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:43:59.727315 7861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9: Bootstrap starting.
I20260806 18:43:59.729739 7861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9: Neither blocks nor log segments found. Creating new log.
I20260806 18:43:59.730536 7861 log.cc:824] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:43:59.730641 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:43:59.729000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55111 (local address 127.0.0.1:37511)
0806 18:43:59.729134 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:43:59.729139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:43:59.729368 (+ 229us) server_negotiation.cc:407] Connection header received
0806 18:43:59.729460 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:43:59.729463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:43:59.729499 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:43:59.729572 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:43:59.730260 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:43:59.730264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:43:59.730266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:43:59.730278 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:43:59.730305 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:43:59.730319 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:43:59.730400 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:43:59.730463 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 18:43:59.730498 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 18:43:59.732369 7861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9: No bootstrap required, opened a new log
I20260806 18:43:59.735720 7861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42adac2ca9d9422db50436030be3b1e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } }
I20260806 18:43:59.735935 7861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:43:59.736004 7861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42adac2ca9d9422db50436030be3b1e9, State: Initialized, Role: FOLLOWER
I20260806 18:43:59.736418 7861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42adac2ca9d9422db50436030be3b1e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } }
I20260806 18:43:59.736563 7861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:43:59.736631 7861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:43:59.736724 7861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:43:59.737569 7861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42adac2ca9d9422db50436030be3b1e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } }
I20260806 18:43:59.737856 7861 leader_election.cc:302] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42adac2ca9d9422db50436030be3b1e9; no voters:
I20260806 18:43:59.738091 7861 leader_election.cc:288] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:43:59.738229 7879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:43:59.738518 7879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [term 1 LEADER]: Becoming Leader. State: Replica: 42adac2ca9d9422db50436030be3b1e9, State: Running, Role: LEADER
I20260806 18:43:59.738873 7879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42adac2ca9d9422db50436030be3b1e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } }
I20260806 18:43:59.739171 7861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:43:59.740381 7882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42adac2ca9d9422db50436030be3b1e9. Latest consensus state: current_term: 1 leader_uuid: "42adac2ca9d9422db50436030be3b1e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42adac2ca9d9422db50436030be3b1e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } } }
I20260806 18:43:59.740496 7882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:43:59.740849 7880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42adac2ca9d9422db50436030be3b1e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42adac2ca9d9422db50436030be3b1e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37511 } } }
I20260806 18:43:59.740949 7880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9 [sys.catalog]: This master's current role is: LEADER
W20260806 18:43:59.744021 7905 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:43:59.744112 7905 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:43:59.744186 7899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:43:59.744906 7899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:43:59.749781 7899 catalog_manager.cc:1347] Generated new cluster ID: b48bccd0a0f54d9ea98e931875766233
I20260806 18:43:59.749846 7899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:43:59.769309 7899 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:43:59.769632 7899 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:43:59.781754 7899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42adac2ca9d9422db50436030be3b1e9: Generated new TSK 0
I20260806 18:43:59.782469 7899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:43:59.796456 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 7758
2026-08-06T18:43:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:43:59 dist-test-slave-k5rr krb5kdc[7913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:43:59 dist-test-slave-k5rr krb5kdc[7913](info): set up 2 sockets
Aug 06 18:43:59 dist-test-slave-k5rr krb5kdc[7913](info): commencing operation
krb5kdc: starting...
W20260806 18:44:00.922024 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:01 dist-test-slave-k5rr krb5kdc[7913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:44:01.048475 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:01.166791 7935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:01.167104 7935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:01.167173 7935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:01.171247 7935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:01.171326 7935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:01.171360 7935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:01.171384 7935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:01.171415 7935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:01.176863 7935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:01.177999 7935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:01.179054 7935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:01.187779 7941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:01.187779 7942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:01.187915 7935 server_base.cc:1034] running on GCE node
W20260806 18:44:01.187781 7944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:01.188732 7935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:01.189908 7935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:01.191073 7935 hybrid_clock.cc:648] HybridClock initialized: now 1786041841191046 us; error 38 us; skew 500 ppm
Aug 06 18:44:01 dist-test-slave-k5rr krb5kdc[7913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041841, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:01.194005 7935 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:44:01.194983 7935 webserver.cc:460] Webserver started at http://127.0.0.1:37463/ using document root <none> and password file <none>
I20260806 18:44:01.195552 7935 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:01.195623 7935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:01.195856 7935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:01.198154 7935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "9088b503dacd4a889ded6c4db55093be"
format_stamp: "Formatted at 2026-08-06 18:44:01 on dist-test-slave-k5rr"
server_key: "de6cae7a8ef7d86e0854499b23c370e2"
server_key_iv: "b4fe5926b968a41dd686e2ff3c861de2"
server_key_version: "encryptionkey@0"
I20260806 18:44:01.198738 7935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "9088b503dacd4a889ded6c4db55093be"
format_stamp: "Formatted at 2026-08-06 18:44:01 on dist-test-slave-k5rr"
server_key: "de6cae7a8ef7d86e0854499b23c370e2"
server_key_iv: "b4fe5926b968a41dd686e2ff3c861de2"
server_key_version: "encryptionkey@0"
I20260806 18:44:01.202695 7935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 18:44:01.205292 7954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:01.206545 7935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:01.206658 7935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "9088b503dacd4a889ded6c4db55093be"
format_stamp: "Formatted at 2026-08-06 18:44:01 on dist-test-slave-k5rr"
server_key: "de6cae7a8ef7d86e0854499b23c370e2"
server_key_iv: "b4fe5926b968a41dd686e2ff3c861de2"
server_key_version: "encryptionkey@0"
I20260806 18:44:01.206799 7935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:01.224534 7935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:01.227770 7935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:01.227959 7935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:01.235414 7935 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35015
I20260806 18:44:01.235522 8032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35015 every 8 connection(s)
I20260806 18:44:01.236438 7935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:01.239337 8034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:01.244993 8034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be: Bootstrap starting.
I20260806 18:44:01.245878 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 7935
I20260806 18:44:01.245997 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:01.246236 9159 external_mini_cluster.cc:1428] Setting key f4468450a4ddf244227e63b109e95ac8
I20260806 18:44:01.247761 8034 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:01.248502 8034 log.cc:824] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:01.250689 8034 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be: No bootstrap required, opened a new log
Aug 06 18:44:01 dist-test-slave-k5rr krb5kdc[7913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:44:01.254685 8034 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9088b503dacd4a889ded6c4db55093be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35015 } }
I20260806 18:44:01.254961 8034 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:01.255030 8034 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9088b503dacd4a889ded6c4db55093be, State: Initialized, Role: FOLLOWER
I20260806 18:44:01.255498 8034 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9088b503dacd4a889ded6c4db55093be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35015 } }
I20260806 18:44:01.255637 8034 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:01.255704 8034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:01.255815 8034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:01.256824 8034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9088b503dacd4a889ded6c4db55093be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35015 } }
I20260806 18:44:01.257189 8034 leader_election.cc:302] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9088b503dacd4a889ded6c4db55093be; no voters:
I20260806 18:44:01.257516 8034 leader_election.cc:288] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:01.257606 8039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:01.257808 8039 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [term 1 LEADER]: Becoming Leader. State: Replica: 9088b503dacd4a889ded6c4db55093be, State: Running, Role: LEADER
I20260806 18:44:01.258267 8039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9088b503dacd4a889ded6c4db55093be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35015 } }
I20260806 18:44:01.258878 8034 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:01.259999 8040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9088b503dacd4a889ded6c4db55093be. Latest consensus state: current_term: 1 leader_uuid: "9088b503dacd4a889ded6c4db55093be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9088b503dacd4a889ded6c4db55093be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35015 } } }
I20260806 18:44:01.260120 8040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:01.260198 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:01.247329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45613 (local address 127.0.0.1:35015)
0806 18:44:01.247782 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:01.247798 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:01.247834 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:44:01.248506 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:01.248523 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:01.248896 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:01.249218 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:01.249893 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:01.251317 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:01.252153 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:01.252863 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:01.253069 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:01.255377 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:01.255382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:01.255396 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:01.255424 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:01.257093 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:01.257532 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:01.257538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:01.257545 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:01.257694 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:01.257965 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:01.257969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:01.257970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:01.258446 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:01.258646 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:01.258900 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 18:44:01.259210 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":124,"threads_started":1}
I20260806 18:44:01.260524 8041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9088b503dacd4a889ded6c4db55093be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9088b503dacd4a889ded6c4db55093be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35015 } } }
I20260806 18:44:01.260708 8041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:01.263258 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 18:44:01.264307 8061 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:01.264387 8061 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:01.264447 8049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:01.265376 8049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:01.271540 8049 catalog_manager.cc:1347] Generated new cluster ID: 5eacbf84a8a24bb3a3f2c1b7e16ef641
I20260806 18:44:01.271615 8049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:01.275072 8037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:01.265267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48583 (local address 127.0.0.1:35015)
0806 18:44:01.265378 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:01.265382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:01.265640 (+ 258us) server_negotiation.cc:407] Connection header received
0806 18:44:01.265742 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:01.265745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:01.265787 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:01.265849 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:01.266474 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:01.266980 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:01.267986 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:01.268392 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:01.268505 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:01.270427 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:01.270431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:01.270434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:01.270460 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:01.273566 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:01.273966 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:01.273970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:01.273972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:01.274051 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:01.274286 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:01.274289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:01.274291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:01.274476 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:01.274588 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:01.274813 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 18:44:01.274920 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:44:01.286952 8049 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:01.288713 8049 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:01.301857 8049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9088b503dacd4a889ded6c4db55093be: Generated new TSK 0
I20260806 18:44:01.302708 8049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:01.308806 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 7935
2026-08-06T18:44:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:01 dist-test-slave-k5rr krb5kdc[8072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:01 dist-test-slave-k5rr krb5kdc[8072](info): set up 2 sockets
Aug 06 18:44:01 dist-test-slave-k5rr krb5kdc[8072](info): commencing operation
krb5kdc: starting...
W20260806 18:44:02.439980 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:02 dist-test-slave-k5rr krb5kdc[8072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:44:02.562364 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41579
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:02.677292 8093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:02.677562 8093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:02.677626 8093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:02.681558 8093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:02.681617 8093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:02.681643 8093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:02.681665 8093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:02.681692 8093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:02.686887 8093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:02.687928 8093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:02.688966 8093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:02.697427 8102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:02.697438 8105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:02.697546 8103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:02.697464 8093 server_base.cc:1034] running on GCE node
I20260806 18:44:02.698163 8093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:02.699203 8093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:02.700392 8093 hybrid_clock.cc:648] HybridClock initialized: now 1786041842700376 us; error 27 us; skew 500 ppm
Aug 06 18:44:02 dist-test-slave-k5rr krb5kdc[8072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041842, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:02.703140 8093 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:44:02.704043 8093 webserver.cc:460] Webserver started at http://127.0.0.1:37793/ using document root <none> and password file <none>
I20260806 18:44:02.704563 8093 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:02.704622 8093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:02.704794 8093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:02.706880 8093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2"
format_stamp: "Formatted at 2026-08-06 18:44:02 on dist-test-slave-k5rr"
server_key: "c87e2466b24659e9bff1032264e47c5c"
server_key_iv: "6c5cf180699bc9220ab9b23a028ffab8"
server_key_version: "encryptionkey@0"
I20260806 18:44:02.707396 8093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2"
format_stamp: "Formatted at 2026-08-06 18:44:02 on dist-test-slave-k5rr"
server_key: "c87e2466b24659e9bff1032264e47c5c"
server_key_iv: "6c5cf180699bc9220ab9b23a028ffab8"
server_key_version: "encryptionkey@0"
I20260806 18:44:02.711103 8093 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:44:02.713481 8115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:02.714634 8093 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:02.714735 8093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2"
format_stamp: "Formatted at 2026-08-06 18:44:02 on dist-test-slave-k5rr"
server_key: "c87e2466b24659e9bff1032264e47c5c"
server_key_iv: "6c5cf180699bc9220ab9b23a028ffab8"
server_key_version: "encryptionkey@0"
I20260806 18:44:02.714819 8093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:02.730681 8093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:02.733892 8093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:02.734050 8093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:02.741158 8093 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41579
I20260806 18:44:02.741185 8199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41579 every 8 connection(s)
I20260806 18:44:02.742097 8093 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:02.744943 8200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:02.749521 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 8093
I20260806 18:44:02.749641 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:02.749881 9159 external_mini_cluster.cc:1428] Setting key e2540e4c986c73c395db29084ece5676
I20260806 18:44:02.750754 8200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2: Bootstrap starting.
I20260806 18:44:02.753122 8200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:02.753903 8200 log.cc:824] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:02.756098 8200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2: No bootstrap required, opened a new log
I20260806 18:44:02.759428 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:02.751190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44803 (local address 127.0.0.1:41579)
0806 18:44:02.751596 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:02.751612 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:02.751648 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:44:02.752297 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:02.752314 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:02.752650 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:02.752934 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:02.753351 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:02.754338 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:02.755317 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:02.755984 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:02.756157 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:02.756899 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:02.756906 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:02.756922 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:02.757474 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:02.757538 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:02.757569 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:02.757664 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:02.757779 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:44:02.757994 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":107,"threads_started":1}
I20260806 18:44:02.760051 8200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260806 18:44:02.760318 8200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:02.760417 8200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd1cbdd899c4b638dace7a3ccda1fb2, State: Initialized, Role: FOLLOWER
I20260806 18:44:02.760880 8200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260806 18:44:02.761046 8200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:02.761114 8200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:02.761216 8200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:02.762243 8200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260806 18:44:02.762434 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:02.762616 8200 leader_election.cc:302] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fd1cbdd899c4b638dace7a3ccda1fb2; no voters:
I20260806 18:44:02.762974 8200 leader_election.cc:288] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:02.763063 8207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:02.763302 8207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [term 1 LEADER]: Becoming Leader. State: Replica: 1fd1cbdd899c4b638dace7a3ccda1fb2, State: Running, Role: LEADER
I20260806 18:44:02.763679 8207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } }
I20260806 18:44:02.764333 8200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:02.765442 8209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd1cbdd899c4b638dace7a3ccda1fb2. Latest consensus state: current_term: 1 leader_uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } } }
I20260806 18:44:02.765579 8209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:02.765997 8208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd1cbdd899c4b638dace7a3ccda1fb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41579 } } }
I20260806 18:44:02.766098 8208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:02.768847 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:02.764608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49015 (local address 127.0.0.1:41579)
0806 18:44:02.764707 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:02.764711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:02.764933 (+ 222us) server_negotiation.cc:407] Connection header received
0806 18:44:02.765154 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:02.765157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:02.765202 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:02.765273 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:02.765653 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:02.766396 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:02.767396 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:02.767622 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:02.767713 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:02.768450 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:02.768454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:02.768456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:02.768469 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:02.768495 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:02.768505 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:02.768564 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:02.768653 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 18:44:02.768709 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 18:44:02.769698 8233 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:02.769790 8233 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:02.769857 8225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:02.770586 8225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:02.775887 8225 catalog_manager.cc:1347] Generated new cluster ID: 4938d05ba30b4e15a059202c8f916842
I20260806 18:44:02.775954 8225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:02.787387 8225 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:02.788311 8225 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:02.800647 8225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fd1cbdd899c4b638dace7a3ccda1fb2: Generated new TSK 0
I20260806 18:44:02.801714 8225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:02.812125 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 8093
2026-08-06T18:44:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:02 dist-test-slave-k5rr krb5kdc[8241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:02 dist-test-slave-k5rr krb5kdc[8241](info): set up 2 sockets
Aug 06 18:44:02 dist-test-slave-k5rr krb5kdc[8241](info): commencing operation
krb5kdc: starting...
W20260806 18:44:03.973425 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:04 dist-test-slave-k5rr krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:44:04.099821 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:04.215034 8267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:04.215373 8267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:04.215436 8267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:04.219563 8267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:04.219626 8267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:04.219645 8267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:04.219659 8267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:04.219686 8267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:04.225098 8267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:04.226219 8267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:04.227353 8267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:04.236472 8276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:04.236483 8277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:04.236483 8280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:04.236589 8267 server_base.cc:1034] running on GCE node
I20260806 18:44:04.237495 8267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:04.238767 8267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:04.239965 8267 hybrid_clock.cc:648] HybridClock initialized: now 1786041844239947 us; error 37 us; skew 500 ppm
Aug 06 18:44:04 dist-test-slave-k5rr krb5kdc[8241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041844, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:04.247879 8267 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:44:04.248878 8267 webserver.cc:460] Webserver started at http://127.0.0.1:38725/ using document root <none> and password file <none>
I20260806 18:44:04.249509 8267 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:04.249581 8267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:04.249804 8267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:04.252017 8267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "92a5526801924f8495cf9b15160a9599"
format_stamp: "Formatted at 2026-08-06 18:44:04 on dist-test-slave-k5rr"
server_key: "12f54b3dad4cbf925dc813385d8ac873"
server_key_iv: "967829180cd195c03e23668f75b70fe9"
server_key_version: "encryptionkey@0"
I20260806 18:44:04.252581 8267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "92a5526801924f8495cf9b15160a9599"
format_stamp: "Formatted at 2026-08-06 18:44:04 on dist-test-slave-k5rr"
server_key: "12f54b3dad4cbf925dc813385d8ac873"
server_key_iv: "967829180cd195c03e23668f75b70fe9"
server_key_version: "encryptionkey@0"
I20260806 18:44:04.256469 8267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:44:04.259083 8289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:04.260298 8267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:04.260421 8267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "92a5526801924f8495cf9b15160a9599"
format_stamp: "Formatted at 2026-08-06 18:44:04 on dist-test-slave-k5rr"
server_key: "12f54b3dad4cbf925dc813385d8ac873"
server_key_iv: "967829180cd195c03e23668f75b70fe9"
server_key_version: "encryptionkey@0"
I20260806 18:44:04.260521 8267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:04.265105 8267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:04.268209 8267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:04.268383 8267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:04.275197 8267 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46491
I20260806 18:44:04.275269 8382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46491 every 8 connection(s)
I20260806 18:44:04.276142 8267 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:04.276888 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 8267
I20260806 18:44:04.277024 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:04.277295 9159 external_mini_cluster.cc:1428] Setting key 38df6117876695b877e2391277a0e259
I20260806 18:44:04.279467 8385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:04.282938 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:04.278676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60089 (local address 127.0.0.1:46491)
0806 18:44:04.279114 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:04.279134 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:04.279179 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:44:04.280049 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:04.280074 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:04.280447 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:04.280843 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:04.281540 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:04.281545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:04.281560 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:04.281571 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:04.281630 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:04.281654 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:04.281792 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:04.281858 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 18:44:04.282012 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":117,"threads_started":1}
I20260806 18:44:04.285918 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:04.286660 8385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599: Bootstrap starting.
I20260806 18:44:04.289525 8385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:04.289865 8388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:04.288229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57321 (local address 127.0.0.1:46491)
0806 18:44:04.288353 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:04.288358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:04.288569 (+ 211us) server_negotiation.cc:407] Connection header received
0806 18:44:04.288662 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:04.288665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:04.288703 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:04.288791 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:04.289481 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:04.289484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:04.289487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:04.289501 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:04.289532 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:04.289546 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:04.289636 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:04.289705 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:44:04.289746 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:44:04.290443 8385 log.cc:824] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:04.292393 8385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599: No bootstrap required, opened a new log
I20260806 18:44:04.295951 8385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a5526801924f8495cf9b15160a9599" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46491 } }
I20260806 18:44:04.296180 8385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:04.296234 8385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92a5526801924f8495cf9b15160a9599, State: Initialized, Role: FOLLOWER
I20260806 18:44:04.296653 8385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a5526801924f8495cf9b15160a9599" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46491 } }
I20260806 18:44:04.296795 8385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:04.296862 8385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:04.296960 8385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:04.297953 8385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a5526801924f8495cf9b15160a9599" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46491 } }
I20260806 18:44:04.298293 8385 leader_election.cc:302] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92a5526801924f8495cf9b15160a9599; no voters:
I20260806 18:44:04.298602 8385 leader_election.cc:288] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:04.298699 8396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:04.298969 8396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [term 1 LEADER]: Becoming Leader. State: Replica: 92a5526801924f8495cf9b15160a9599, State: Running, Role: LEADER
I20260806 18:44:04.299327 8396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a5526801924f8495cf9b15160a9599" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46491 } }
I20260806 18:44:04.299794 8385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:04.301055 8397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92a5526801924f8495cf9b15160a9599" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a5526801924f8495cf9b15160a9599" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46491 } } }
I20260806 18:44:04.301055 8398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92a5526801924f8495cf9b15160a9599. Latest consensus state: current_term: 1 leader_uuid: "92a5526801924f8495cf9b15160a9599" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92a5526801924f8495cf9b15160a9599" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46491 } } }
I20260806 18:44:04.301239 8397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:04.301249 8398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:04.301669 8410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:04.304565 8410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:04.310446 8410 catalog_manager.cc:1347] Generated new cluster ID: a247c75f85ff4118b64745eafcbba215
I20260806 18:44:04.310513 8410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:04.319377 8410 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:04.319689 8410 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:04.332976 8410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92a5526801924f8495cf9b15160a9599: Generated new TSK 0
I20260806 18:44:04.333905 8410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:04.342356 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 8267
2026-08-06T18:44:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1542 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:04 dist-test-slave-k5rr krb5kdc[8431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:04 dist-test-slave-k5rr krb5kdc[8431](info): set up 2 sockets
Aug 06 18:44:04 dist-test-slave-k5rr krb5kdc[8431](info): commencing operation
krb5kdc: starting...
W20260806 18:44:05.498575 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:05 dist-test-slave-k5rr krb5kdc[8431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:44:05.630223 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:05.748059 8452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:05.748368 8452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:05.748410 8452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:05.752446 8452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:05.752509 8452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:05.752527 8452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:05.752544 8452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:05.752573 8452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:05.758016 8452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:05.759136 8452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:05.760210 8452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:05.768728 8460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:05.768765 8462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:05.768772 8459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:05.768786 8452 server_base.cc:1034] running on GCE node
I20260806 18:44:05.769677 8452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:05.770856 8452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:05.772056 8452 hybrid_clock.cc:648] HybridClock initialized: now 1786041845772037 us; error 37 us; skew 500 ppm
Aug 06 18:44:05 dist-test-slave-k5rr krb5kdc[8431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041845, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:05.774906 8452 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:44:05.775841 8452 webserver.cc:460] Webserver started at http://127.0.0.1:39435/ using document root <none> and password file <none>
I20260806 18:44:05.776412 8452 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:05.776481 8452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:05.776701 8452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:05.778893 8452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "55f5270bfb6149488f98acd5deb04795"
format_stamp: "Formatted at 2026-08-06 18:44:05 on dist-test-slave-k5rr"
server_key: "d77de27b16cb78d2981e75f157ea0e88"
server_key_iv: "854c6513883c2536add3121ee56d7a12"
server_key_version: "encryptionkey@0"
I20260806 18:44:05.779479 8452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "55f5270bfb6149488f98acd5deb04795"
format_stamp: "Formatted at 2026-08-06 18:44:05 on dist-test-slave-k5rr"
server_key: "d77de27b16cb78d2981e75f157ea0e88"
server_key_iv: "854c6513883c2536add3121ee56d7a12"
server_key_version: "encryptionkey@0"
I20260806 18:44:05.783320 8452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:44:05.785848 8476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:05.787127 8452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:05.787268 8452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "55f5270bfb6149488f98acd5deb04795"
format_stamp: "Formatted at 2026-08-06 18:44:05 on dist-test-slave-k5rr"
server_key: "d77de27b16cb78d2981e75f157ea0e88"
server_key_iv: "854c6513883c2536add3121ee56d7a12"
server_key_version: "encryptionkey@0"
I20260806 18:44:05.787372 8452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:05.825430 8452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:05.828828 8452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:05.829073 8452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:05.836571 8452 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45617
I20260806 18:44:05.836592 8550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45617 every 8 connection(s)
I20260806 18:44:05.837661 8452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:05.838145 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 8452
I20260806 18:44:05.838301 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:05.838593 9159 external_mini_cluster.cc:1428] Setting key fd57c8513ce152f8b2345fdb7dc024a2
I20260806 18:44:05.841094 8551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:05.847040 8551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795: Bootstrap starting.
Aug 06 18:44:05 dist-test-slave-k5rr krb5kdc[8431](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:44:05.849560 8551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:05.850346 8551 log.cc:824] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:05.852419 8551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795: No bootstrap required, opened a new log
I20260806 18:44:05.852967 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:05.840178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:58043 (local address 127.0.0.1:45617)
0806 18:44:05.840680 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:05.840696 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:05.840737 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:44:05.841553 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:05.841576 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:05.841951 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:05.842303 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:05.843166 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:05.844206 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:05.845139 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:05.845614 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:05.845760 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:05.848311 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:05.848316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:05.848330 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:05.848360 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:05.850033 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:05.850532 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:05.850538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:05.850543 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:05.850654 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:05.850989 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:05.850992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:05.850993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:05.851413 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:05.851623 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:05.851968 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 18:44:05.852198 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":178,"threads_started":1}
I20260806 18:44:05.856168 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:05.856570 8551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5270bfb6149488f98acd5deb04795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45617 } }
I20260806 18:44:05.856802 8551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:05.856897 8551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f5270bfb6149488f98acd5deb04795, State: Initialized, Role: FOLLOWER
I20260806 18:44:05.857404 8551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5270bfb6149488f98acd5deb04795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45617 } }
I20260806 18:44:05.857569 8551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:05.857638 8551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:05.857738 8551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:05.858742 8551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5270bfb6149488f98acd5deb04795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45617 } }
I20260806 18:44:05.859090 8551 leader_election.cc:302] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55f5270bfb6149488f98acd5deb04795; no voters:
I20260806 18:44:05.859371 8551 leader_election.cc:288] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:05.859489 8568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:05.859712 8568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [term 1 LEADER]: Becoming Leader. State: Replica: 55f5270bfb6149488f98acd5deb04795, State: Running, Role: LEADER
I20260806 18:44:05.860060 8568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5270bfb6149488f98acd5deb04795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45617 } }
I20260806 18:44:05.860651 8551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:05.861816 8569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55f5270bfb6149488f98acd5deb04795" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5270bfb6149488f98acd5deb04795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45617 } } }
I20260806 18:44:05.861929 8569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:05.862324 8581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:05.863271 8570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55f5270bfb6149488f98acd5deb04795. Latest consensus state: current_term: 1 leader_uuid: "55f5270bfb6149488f98acd5deb04795" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5270bfb6149488f98acd5deb04795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45617 } } }
I20260806 18:44:05.863394 8570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:05.865731 8581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:05.867555 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:05.858257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:48981 (local address 127.0.0.1:45617)
0806 18:44:05.858387 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:05.858391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:05.858598 (+ 207us) server_negotiation.cc:407] Connection header received
0806 18:44:05.858707 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:05.858710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:05.858755 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:05.858828 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:05.859521 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:05.860259 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:05.862173 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:05.862498 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:05.862593 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:05.864061 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:05.864065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:05.864068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:05.864094 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:05.865705 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:05.866257 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:05.866262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:05.866264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:05.866351 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:05.866729 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:05.866732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:05.866734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:05.866913 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:05.867044 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:05.867308 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 18:44:05.867409 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:44:05.872085 8581 catalog_manager.cc:1347] Generated new cluster ID: 874115623b464a1482f67e02e3492931
I20260806 18:44:05.872197 8581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:05.885392 8581 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:05.886477 8581 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:05.894158 8581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55f5270bfb6149488f98acd5deb04795: Generated new TSK 0
I20260806 18:44:05.894981 8581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:05.905859 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 8452
W20260806 18:44:05.917840 8428 connection.cc:535] client connection to 127.0.0.1:45617 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45617): Connection reset by peer (error 104)
2026-08-06T18:44:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:05 dist-test-slave-k5rr krb5kdc[8603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:05 dist-test-slave-k5rr krb5kdc[8603](info): set up 2 sockets
Aug 06 18:44:05 dist-test-slave-k5rr krb5kdc[8603](info): commencing operation
krb5kdc: starting...
W20260806 18:44:07.065317 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:07 dist-test-slave-k5rr krb5kdc[8603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:44:07.186175 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:07.302615 8622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:07.302874 8622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:07.302932 8622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:07.306851 8622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:07.306905 8622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:07.306931 8622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:07.306953 8622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:07.306981 8622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:07.312240 8622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:07.313367 8622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:07.314345 8622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:07.322731 8630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:07.322703 8629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:07.322702 8632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:07.322893 8622 server_base.cc:1034] running on GCE node
I20260806 18:44:07.323508 8622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:07.324496 8622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:07.325640 8622 hybrid_clock.cc:648] HybridClock initialized: now 1786041847325620 us; error 32 us; skew 500 ppm
Aug 06 18:44:07 dist-test-slave-k5rr krb5kdc[8603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041847, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:07.328246 8622 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:44:07.329160 8622 webserver.cc:460] Webserver started at http://127.0.0.1:46811/ using document root <none> and password file <none>
I20260806 18:44:07.329695 8622 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:07.329761 8622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:07.329972 8622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:07.331986 8622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "d8bed59b3b84498a96fec9b285af5725"
format_stamp: "Formatted at 2026-08-06 18:44:07 on dist-test-slave-k5rr"
server_key: "75bec612126b25e445204c5322390997"
server_key_iv: "61ad531fb4a9d8fdbb8fda51b38e06ce"
server_key_version: "encryptionkey@0"
I20260806 18:44:07.332531 8622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "d8bed59b3b84498a96fec9b285af5725"
format_stamp: "Formatted at 2026-08-06 18:44:07 on dist-test-slave-k5rr"
server_key: "75bec612126b25e445204c5322390997"
server_key_iv: "61ad531fb4a9d8fdbb8fda51b38e06ce"
server_key_version: "encryptionkey@0"
I20260806 18:44:07.336058 8622 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:44:07.338438 8647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:07.339607 8622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:44:07.339720 8622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "d8bed59b3b84498a96fec9b285af5725"
format_stamp: "Formatted at 2026-08-06 18:44:07 on dist-test-slave-k5rr"
server_key: "75bec612126b25e445204c5322390997"
server_key_iv: "61ad531fb4a9d8fdbb8fda51b38e06ce"
server_key_version: "encryptionkey@0"
I20260806 18:44:07.339816 8622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:07.354902 8622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:07.357667 8622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:07.357843 8622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:07.364954 8622 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44039
I20260806 18:44:07.364980 8718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44039 every 8 connection(s)
I20260806 18:44:07.365931 8622 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:07.368700 8719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:07.372330 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 8622
I20260806 18:44:07.372462 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:07.372735 9159 external_mini_cluster.cc:1428] Setting key 5f94ec3838410fce6f0a6679081323bd
I20260806 18:44:07.374622 8719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725: Bootstrap starting.
I20260806 18:44:07.377429 8719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:07.378321 8719 log.cc:824] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:07.380599 8719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725: No bootstrap required, opened a new log
I20260806 18:44:07.383312 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:07.374127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:45323 (local address 127.0.0.1:44039)
0806 18:44:07.374550 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:07.374570 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:07.374612 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:44:07.375426 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:07.375450 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:07.375892 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:07.376258 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:07.376702 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:07.378194 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:07.379348 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:07.380024 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:07.380192 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:07.380993 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:07.381000 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:07.381045 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:07.381538 (+ 493us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:07.381608 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:07.381641 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:07.381775 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:07.381985 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 18:44:07.382123 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":116,"threads_started":1}
I20260806 18:44:07.384388 8719 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bed59b3b84498a96fec9b285af5725" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } }
I20260806 18:44:07.384655 8719 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:07.384725 8719 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8bed59b3b84498a96fec9b285af5725, State: Initialized, Role: FOLLOWER
I20260806 18:44:07.385200 8719 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bed59b3b84498a96fec9b285af5725" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } }
I20260806 18:44:07.385352 8719 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:07.385419 8719 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:07.385514 8719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:07.386332 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:07.386591 8719 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bed59b3b84498a96fec9b285af5725" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } }
I20260806 18:44:07.386968 8719 leader_election.cc:302] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8bed59b3b84498a96fec9b285af5725; no voters:
I20260806 18:44:07.387334 8719 leader_election.cc:288] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:07.387418 8727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:07.387679 8727 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [term 1 LEADER]: Becoming Leader. State: Replica: d8bed59b3b84498a96fec9b285af5725, State: Running, Role: LEADER
I20260806 18:44:07.388027 8727 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bed59b3b84498a96fec9b285af5725" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } }
I20260806 18:44:07.388584 8719 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:07.389146 8727 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8bed59b3b84498a96fec9b285af5725. Latest consensus state: current_term: 1 leader_uuid: "d8bed59b3b84498a96fec9b285af5725" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bed59b3b84498a96fec9b285af5725" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } } }
I20260806 18:44:07.389292 8727 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:07.389680 8728 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8bed59b3b84498a96fec9b285af5725" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bed59b3b84498a96fec9b285af5725" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44039 } } }
I20260806 18:44:07.389775 8728 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:07.390072 8741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:07.393576 8741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:07.395905 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:07.391252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:36559 (local address 127.0.0.1:44039)
0806 18:44:07.391354 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:07.391359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:07.391620 (+ 261us) server_negotiation.cc:407] Connection header received
0806 18:44:07.391846 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:07.391849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:07.391902 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:07.391985 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:07.392275 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:07.393035 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:07.394123 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:07.394464 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:07.394575 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:07.395491 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:07.395496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:07.395498 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:07.395512 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:07.395540 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:07.395554 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:07.395643 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:07.395741 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:44:07.395776 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:44:07.401113 8741 catalog_manager.cc:1347] Generated new cluster ID: a3c1d703aaec409d95d269b46c10ecde
I20260806 18:44:07.401167 8741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:07.418078 8741 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:07.419238 8741 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:07.428694 8741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8bed59b3b84498a96fec9b285af5725: Generated new TSK 0
I20260806 18:44:07.429561 8741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:07.433732 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 8622
W20260806 18:44:07.449127 8598 connection.cc:535] client connection to 127.0.0.1:44039 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44039): Connection reset by peer (error 104)
2026-08-06T18:44:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1531 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:07 dist-test-slave-k5rr krb5kdc[8765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:07 dist-test-slave-k5rr krb5kdc[8765](info): set up 2 sockets
Aug 06 18:44:07 dist-test-slave-k5rr krb5kdc[8765](info): commencing operation
krb5kdc: starting...
W20260806 18:44:08.564062 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:08 dist-test-slave-k5rr krb5kdc[8765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:44:08.683619 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38677
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:08.795006 8786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:08.795288 8786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:08.795348 8786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:08.799275 8786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:08.799333 8786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:08.799360 8786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:08.799381 8786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:08.799401 8786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:08.804702 8786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:08.805743 8786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:08.806751 8786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:08.815302 8797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:08.815394 8786 server_base.cc:1034] running on GCE node
W20260806 18:44:08.815331 8795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:08.815296 8794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:08.816144 8786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:08.817301 8786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:08.818464 8786 hybrid_clock.cc:648] HybridClock initialized: now 1786041848818448 us; error 29 us; skew 500 ppm
Aug 06 18:44:08 dist-test-slave-k5rr krb5kdc[8765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:08.821149 8786 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:44:08.822000 8786 webserver.cc:460] Webserver started at http://127.0.0.1:33825/ using document root <none> and password file <none>
I20260806 18:44:08.822525 8786 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:08.822590 8786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:08.822793 8786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:08.824812 8786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "0b29af36aed04ec1879b287b9692dd85"
format_stamp: "Formatted at 2026-08-06 18:44:08 on dist-test-slave-k5rr"
server_key: "a3943c910dd6660e53bf99ab012727f6"
server_key_iv: "01eb13075e55bc83dd01acc3f8cdf4f6"
server_key_version: "encryptionkey@0"
I20260806 18:44:08.825392 8786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "0b29af36aed04ec1879b287b9692dd85"
format_stamp: "Formatted at 2026-08-06 18:44:08 on dist-test-slave-k5rr"
server_key: "a3943c910dd6660e53bf99ab012727f6"
server_key_iv: "01eb13075e55bc83dd01acc3f8cdf4f6"
server_key_version: "encryptionkey@0"
I20260806 18:44:08.829032 8786 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:44:08.831404 8807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:08.832625 8786 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:08.832736 8786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "0b29af36aed04ec1879b287b9692dd85"
format_stamp: "Formatted at 2026-08-06 18:44:08 on dist-test-slave-k5rr"
server_key: "a3943c910dd6660e53bf99ab012727f6"
server_key_iv: "01eb13075e55bc83dd01acc3f8cdf4f6"
server_key_version: "encryptionkey@0"
I20260806 18:44:08.832827 8786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:08.837255 8786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:08.839970 8786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:08.840133 8786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:08.846743 8786 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38677
I20260806 18:44:08.846786 8875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38677 every 8 connection(s)
I20260806 18:44:08.847654 8786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:08.849630 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 8786
I20260806 18:44:08.849757 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:08.850040 9159 external_mini_cluster.cc:1428] Setting key 89be16bb27fc4c247995b3812b0d0ddc
I20260806 18:44:08.850858 8876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:08.855782 8882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:08.851405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:60269 (local address 127.0.0.1:38677)
0806 18:44:08.851859 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:08.851880 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:08.851925 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:44:08.852811 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:08.852835 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:08.853228 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:08.853611 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:08.854282 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:08.854288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:08.854304 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:08.854317 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:08.854380 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:08.854410 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:08.854566 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:08.854639 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 18:44:08.854807 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":120,"threads_started":1}
I20260806 18:44:08.858140 8876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85: Bootstrap starting.
I20260806 18:44:08.858709 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:08.860958 8876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:08.861758 8876 log.cc:824] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:08.862600 8882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:08.861097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:37193 (local address 127.0.0.1:38677)
0806 18:44:08.861217 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:08.861221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:08.861420 (+ 199us) server_negotiation.cc:407] Connection header received
0806 18:44:08.861521 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:08.861525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:08.861554 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:08.861616 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:08.862251 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:08.862255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:08.862258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:08.862269 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:08.862296 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:08.862307 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:08.862376 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:08.862456 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 18:44:08.862497 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:44:08.863855 8876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85: No bootstrap required, opened a new log
I20260806 18:44:08.867576 8876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b29af36aed04ec1879b287b9692dd85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38677 } }
I20260806 18:44:08.867817 8876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:08.867865 8876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b29af36aed04ec1879b287b9692dd85, State: Initialized, Role: FOLLOWER
I20260806 18:44:08.868256 8876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b29af36aed04ec1879b287b9692dd85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38677 } }
I20260806 18:44:08.868399 8876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:08.868450 8876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:08.868526 8876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:08.869407 8876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b29af36aed04ec1879b287b9692dd85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38677 } }
I20260806 18:44:08.869722 8876 leader_election.cc:302] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b29af36aed04ec1879b287b9692dd85; no voters:
I20260806 18:44:08.870002 8876 leader_election.cc:288] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:08.870100 8893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:08.870317 8893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [term 1 LEADER]: Becoming Leader. State: Replica: 0b29af36aed04ec1879b287b9692dd85, State: Running, Role: LEADER
I20260806 18:44:08.870668 8893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b29af36aed04ec1879b287b9692dd85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38677 } }
I20260806 18:44:08.871177 8876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:08.872284 8896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b29af36aed04ec1879b287b9692dd85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b29af36aed04ec1879b287b9692dd85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38677 } } }
I20260806 18:44:08.872428 8896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:08.872749 8899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b29af36aed04ec1879b287b9692dd85. Latest consensus state: current_term: 1 leader_uuid: "0b29af36aed04ec1879b287b9692dd85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b29af36aed04ec1879b287b9692dd85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38677 } } }
I20260806 18:44:08.872844 8899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:08.873297 8908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:08.875932 8908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:08.881340 8908 catalog_manager.cc:1347] Generated new cluster ID: 62372c7568c64f9091b38413383e9464
I20260806 18:44:08.881420 8908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:08.887656 8908 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:08.887977 8908 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:08.897367 8908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b29af36aed04ec1879b287b9692dd85: Generated new TSK 0
I20260806 18:44:08.898146 8908 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:08.905673 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 8786
2026-08-06T18:44:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1475 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:08 dist-test-slave-k5rr krb5kdc[8927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:08 dist-test-slave-k5rr krb5kdc[8927](info): set up 2 sockets
Aug 06 18:44:08 dist-test-slave-k5rr krb5kdc[8927](info): commencing operation
krb5kdc: starting...
W20260806 18:44:10.046276 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:10 dist-test-slave-k5rr krb5kdc[8927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:44:10.171411 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:10.285774 8945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:10.286059 8945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:10.286101 8945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:10.290174 8945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:10.290241 8945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:10.290259 8945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:10.290277 8945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:10.290289 8945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:10.295766 8945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42703
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:10.296830 8945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:10.297955 8945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:10.306155 8961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:10.306139 8955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:10.306316 8945 server_base.cc:1034] running on GCE node
W20260806 18:44:10.306471 8957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:10.307065 8945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:10.308172 8945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:10.309325 8945 hybrid_clock.cc:648] HybridClock initialized: now 1786041850309308 us; error 35 us; skew 500 ppm
Aug 06 18:44:10 dist-test-slave-k5rr krb5kdc[8927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041850, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:10.316941 8945 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:44:10.317986 8945 webserver.cc:460] Webserver started at http://127.0.0.1:43769/ using document root <none> and password file <none>
I20260806 18:44:10.318567 8945 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:10.318629 8945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:10.318840 8945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:10.320971 8945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "10fde63256b1498181e02abf51a1cae9"
format_stamp: "Formatted at 2026-08-06 18:44:10 on dist-test-slave-k5rr"
server_key: "d35fb1e780eae5d3190d1d24059317c8"
server_key_iv: "323d6556ea178acdd86c91e8c5b2b3ac"
server_key_version: "encryptionkey@0"
I20260806 18:44:10.321581 8945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "10fde63256b1498181e02abf51a1cae9"
format_stamp: "Formatted at 2026-08-06 18:44:10 on dist-test-slave-k5rr"
server_key: "d35fb1e780eae5d3190d1d24059317c8"
server_key_iv: "323d6556ea178acdd86c91e8c5b2b3ac"
server_key_version: "encryptionkey@0"
I20260806 18:44:10.325474 8945 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:44:10.327986 8971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:10.329327 8945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:10.329452 8945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "10fde63256b1498181e02abf51a1cae9"
format_stamp: "Formatted at 2026-08-06 18:44:10 on dist-test-slave-k5rr"
server_key: "d35fb1e780eae5d3190d1d24059317c8"
server_key_iv: "323d6556ea178acdd86c91e8c5b2b3ac"
server_key_version: "encryptionkey@0"
I20260806 18:44:10.329550 8945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:10.350834 8945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:10.353940 8945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:10.354143 8945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:10.361833 8945 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42703
I20260806 18:44:10.361948 9051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42703 every 8 connection(s)
I20260806 18:44:10.363116 8945 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:10.366492 9053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:10.368367 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 8945
I20260806 18:44:10.368517 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:10.368784 9159 external_mini_cluster.cc:1428] Setting key f9759bcdaac0cff93327370e2fb93de2
I20260806 18:44:10.373744 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9: Bootstrap starting.
I20260806 18:44:10.376587 9053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:10.377470 9053 log.cc:824] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:44:10 dist-test-slave-k5rr krb5kdc[8927](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:44:10.379676 9053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9: No bootstrap required, opened a new log
I20260806 18:44:10.384382 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:10.370226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:51923 (local address 127.0.0.1:42703)
0806 18:44:10.370703 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:10.370729 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:10.370873 (+ 144us) server_negotiation.cc:407] Connection header received
0806 18:44:10.371706 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:10.371733 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:10.372209 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:10.372612 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:10.373423 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:10.374865 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:10.375768 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:10.376490 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:10.376665 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:10.379180 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:10.379187 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:10.379205 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:10.379237 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:10.381184 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:10.381667 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:10.381674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:10.381680 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:10.381824 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:10.382121 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:10.382124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:10.382126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:10.382665 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:10.382872 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:10.383119 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 18:44:10.383355 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":144,"threads_started":1}
I20260806 18:44:10.384377 9053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fde63256b1498181e02abf51a1cae9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42703 } }
I20260806 18:44:10.384824 9053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:10.384887 9053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fde63256b1498181e02abf51a1cae9, State: Initialized, Role: FOLLOWER
I20260806 18:44:10.385465 9053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fde63256b1498181e02abf51a1cae9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42703 } }
I20260806 18:44:10.385644 9053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:10.385713 9053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:10.385805 9053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:10.386809 9053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fde63256b1498181e02abf51a1cae9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42703 } }
I20260806 18:44:10.387198 9053 leader_election.cc:302] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10fde63256b1498181e02abf51a1cae9; no voters:
I20260806 18:44:10.387579 9053 leader_election.cc:288] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:10.387712 9062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:10.387849 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:10.387995 9062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [term 1 LEADER]: Becoming Leader. State: Replica: 10fde63256b1498181e02abf51a1cae9, State: Running, Role: LEADER
I20260806 18:44:10.388345 9062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fde63256b1498181e02abf51a1cae9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42703 } }
I20260806 18:44:10.388923 9053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:10.390038 9064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10fde63256b1498181e02abf51a1cae9. Latest consensus state: current_term: 1 leader_uuid: "10fde63256b1498181e02abf51a1cae9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fde63256b1498181e02abf51a1cae9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42703 } } }
I20260806 18:44:10.390172 9064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:10.390507 9063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10fde63256b1498181e02abf51a1cae9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10fde63256b1498181e02abf51a1cae9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42703 } } }
I20260806 18:44:10.390609 9063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:10.390931 9082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:10.394289 9082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:10.400094 9082 catalog_manager.cc:1347] Generated new cluster ID: ee5c094e225c492e9332493bdc8d2a07
I20260806 18:44:10.400157 9082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:10.401476 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:10.390189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:58737 (local address 127.0.0.1:42703)
0806 18:44:10.390293 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:10.390297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:10.393277 (+ 2980us) server_negotiation.cc:407] Connection header received
0806 18:44:10.393395 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:10.393398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:10.393450 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:10.393539 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:10.394330 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:10.395135 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:10.396212 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:10.396473 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:10.396560 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:10.397873 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:10.397876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:10.397878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:10.397900 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:10.399535 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:10.400097 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:10.400100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:10.400102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:10.400291 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:10.400616 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:10.400618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:10.400620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:10.400802 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:10.400945 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:10.401252 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 18:44:10.401344 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:44:10.408407 9082 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:10.409361 9082 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:10.413614 9082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10fde63256b1498181e02abf51a1cae9: Generated new TSK 0
I20260806 18:44:10.414252 9082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:10.419616 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 8945
W20260806 18:44:10.432313 8922 connection.cc:535] client connection to 127.0.0.1:42703 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42703): Connection reset by peer (error 104)
2026-08-06T18:44:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:10 dist-test-slave-k5rr krb5kdc[9101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:10 dist-test-slave-k5rr krb5kdc[9101](info): set up 2 sockets
Aug 06 18:44:10 dist-test-slave-k5rr krb5kdc[9101](info): commencing operation
krb5kdc: starting...
W20260806 18:44:11.565261 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:11 dist-test-slave-k5rr krb5kdc[9101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:44:11.687879 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43809
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:11.800549 9117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:11.800813 9117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:11.800874 9117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:11.804759 9117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:11.804823 9117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:11.804852 9117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:11.804872 9117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:11.804901 9117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:11.810135 9117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43809
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:11.811183 9117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:11.812212 9117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:11.820465 9125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:11.820508 9131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:11.820602 9117 server_base.cc:1034] running on GCE node
W20260806 18:44:11.820470 9126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:11.821336 9117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:11.822377 9117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:11.823529 9117 hybrid_clock.cc:648] HybridClock initialized: now 1786041851823512 us; error 27 us; skew 500 ppm
Aug 06 18:44:11 dist-test-slave-k5rr krb5kdc[9101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:11.826261 9117 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:44:11.827173 9117 webserver.cc:460] Webserver started at http://127.0.0.1:34165/ using document root <none> and password file <none>
I20260806 18:44:11.827728 9117 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:11.827795 9117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:11.827999 9117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:11.830086 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "6d9b21a3d72b429fb6e8006104df86b2"
format_stamp: "Formatted at 2026-08-06 18:44:11 on dist-test-slave-k5rr"
server_key: "2eee2b79d1025d986656335f52d48f29"
server_key_iv: "01beafd4ec315ac72d08f0f22a473d9e"
server_key_version: "encryptionkey@0"
I20260806 18:44:11.830632 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "6d9b21a3d72b429fb6e8006104df86b2"
format_stamp: "Formatted at 2026-08-06 18:44:11 on dist-test-slave-k5rr"
server_key: "2eee2b79d1025d986656335f52d48f29"
server_key_iv: "01beafd4ec315ac72d08f0f22a473d9e"
server_key_version: "encryptionkey@0"
I20260806 18:44:11.834355 9117 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:44:11.836695 9143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:11.837946 9117 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:11.838063 9117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "6d9b21a3d72b429fb6e8006104df86b2"
format_stamp: "Formatted at 2026-08-06 18:44:11 on dist-test-slave-k5rr"
server_key: "2eee2b79d1025d986656335f52d48f29"
server_key_iv: "01beafd4ec315ac72d08f0f22a473d9e"
server_key_version: "encryptionkey@0"
I20260806 18:44:11.838160 9117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:11.853202 9117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:11.856140 9117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:11.856343 9117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:11.863469 9117 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43809
I20260806 18:44:11.863484 9225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43809 every 8 connection(s)
I20260806 18:44:11.864464 9117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:11.867316 9226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:11.872856 9226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2: Bootstrap starting.
I20260806 18:44:11.874426 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9117
I20260806 18:44:11.874552 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:11.874816 9159 external_mini_cluster.cc:1428] Setting key 04c40153fb2877b24c7c197578fea503
I20260806 18:44:11.875504 9226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:11.876341 9226 log.cc:824] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:11.878583 9226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2: No bootstrap required, opened a new log
I20260806 18:44:11.882717 9226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b21a3d72b429fb6e8006104df86b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } }
I20260806 18:44:11.882994 9226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:11.883055 9226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d9b21a3d72b429fb6e8006104df86b2, State: Initialized, Role: FOLLOWER
I20260806 18:44:11.883553 9226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b21a3d72b429fb6e8006104df86b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } }
I20260806 18:44:11.883714 9226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:11.883782 9226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:11.883876 9226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:11.884678 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:11.875896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:33993 (local address 127.0.0.1:43809)
0806 18:44:11.876372 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:11.876393 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:11.876437 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:44:11.877245 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:11.877269 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:11.877686 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:11.878055 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:11.878490 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:11.879919 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:11.880798 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:11.881571 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:11.881734 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:11.882658 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:11.882664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:11.882680 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:11.883194 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:11.883262 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:11.883294 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:11.883419 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:11.883572 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 18:44:11.883714 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":144,"threads_started":1}
I20260806 18:44:11.885056 9226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b21a3d72b429fb6e8006104df86b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } }
I20260806 18:44:11.885391 9226 leader_election.cc:302] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d9b21a3d72b429fb6e8006104df86b2; no voters:
I20260806 18:44:11.885676 9226 leader_election.cc:288] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:11.885753 9233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:11.885939 9233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [term 1 LEADER]: Becoming Leader. State: Replica: 6d9b21a3d72b429fb6e8006104df86b2, State: Running, Role: LEADER
I20260806 18:44:11.886230 9233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b21a3d72b429fb6e8006104df86b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } }
I20260806 18:44:11.886977 9226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:11.887665 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:11.887972 9234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d9b21a3d72b429fb6e8006104df86b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b21a3d72b429fb6e8006104df86b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } } }
I20260806 18:44:11.888111 9234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:11.888428 9235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d9b21a3d72b429fb6e8006104df86b2. Latest consensus state: current_term: 1 leader_uuid: "6d9b21a3d72b429fb6e8006104df86b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d9b21a3d72b429fb6e8006104df86b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43809 } } }
I20260806 18:44:11.888525 9235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2 [sys.catalog]: This master's current role is: LEADER
W20260806 18:44:11.892123 9258 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:11.892204 9258 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:11.892287 9244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:11.893042 9244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:11.894784 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:11.889896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:44103 (local address 127.0.0.1:43809)
0806 18:44:11.890118 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:11.890122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:11.890164 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:44:11.890286 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:11.890289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:11.890324 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:11.890385 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:11.890657 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:11.891418 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:11.892474 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:11.892799 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:11.892914 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:11.894070 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:11.894074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:11.894077 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:11.894090 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:11.894118 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:11.894131 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:11.894221 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:11.894613 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 18:44:11.894651 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260806 18:44:11.899202 9244 catalog_manager.cc:1347] Generated new cluster ID: 4920524a2b3f470f95388e9a52fbda9c
I20260806 18:44:11.899267 9244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:11.905283 9244 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:11.906694 9244 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:11.916330 9244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d9b21a3d72b429fb6e8006104df86b2: Generated new TSK 0
I20260806 18:44:11.917081 9244 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:11.923318 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9117
W20260806 18:44:11.935817 9097 connection.cc:535] client connection to 127.0.0.1:43809 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43809): Connection reset by peer (error 104)
2026-08-06T18:44:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:11 dist-test-slave-k5rr krb5kdc[9272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:11 dist-test-slave-k5rr krb5kdc[9272](info): set up 2 sockets
Aug 06 18:44:11 dist-test-slave-k5rr krb5kdc[9272](info): commencing operation
krb5kdc: starting...
W20260806 18:44:13.062383 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:13 dist-test-slave-k5rr krb5kdc[9272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:44:13.194296 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:13.312570 9287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:13.312942 9287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:13.313045 9287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:13.317281 9287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:13.317338 9287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:13.317364 9287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:13.317387 9287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:13.317416 9287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:13.322880 9287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38599
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:13.323917 9287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:13.324926 9287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:13.333266 9297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:13.333328 9298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:13.333339 9287 server_base.cc:1034] running on GCE node
W20260806 18:44:13.333473 9300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:13.334008 9287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:13.335021 9287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:13.336185 9287 hybrid_clock.cc:648] HybridClock initialized: now 1786041853336171 us; error 24 us; skew 500 ppm
Aug 06 18:44:13 dist-test-slave-k5rr krb5kdc[9272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041853, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:13.338761 9287 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:44:13.339570 9287 webserver.cc:460] Webserver started at http://127.0.0.1:34317/ using document root <none> and password file <none>
I20260806 18:44:13.340070 9287 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:13.340126 9287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:13.340292 9287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:13.342280 9287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "94d6b014b3e4407396a2a8f35e6c54a0"
format_stamp: "Formatted at 2026-08-06 18:44:13 on dist-test-slave-k5rr"
server_key: "1d6528c4d78fa05579b8466315f142b2"
server_key_iv: "309561a7098d7e38f8e3f9f43ec79780"
server_key_version: "encryptionkey@0"
I20260806 18:44:13.342777 9287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "94d6b014b3e4407396a2a8f35e6c54a0"
format_stamp: "Formatted at 2026-08-06 18:44:13 on dist-test-slave-k5rr"
server_key: "1d6528c4d78fa05579b8466315f142b2"
server_key_iv: "309561a7098d7e38f8e3f9f43ec79780"
server_key_version: "encryptionkey@0"
I20260806 18:44:13.346311 9287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:44:13.348542 9311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:13.349882 9287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:13.349990 9287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "94d6b014b3e4407396a2a8f35e6c54a0"
format_stamp: "Formatted at 2026-08-06 18:44:13 on dist-test-slave-k5rr"
server_key: "1d6528c4d78fa05579b8466315f142b2"
server_key_iv: "309561a7098d7e38f8e3f9f43ec79780"
server_key_version: "encryptionkey@0"
I20260806 18:44:13.350081 9287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:13.354526 9287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:13.357313 9287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:13.357475 9287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:13.364203 9287 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38599
I20260806 18:44:13.364280 9384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38599 every 8 connection(s)
I20260806 18:44:13.365170 9287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:13.367992 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:13.371090 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9287
I20260806 18:44:13.371233 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:13.371567 9159 external_mini_cluster.cc:1428] Setting key 374f02eefda58a7f53926c493fdb6898
I20260806 18:44:13.374487 9385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0: Bootstrap starting.
I20260806 18:44:13.377355 9385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:13.378300 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:13.373199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:45903 (local address 127.0.0.1:38599)
0806 18:44:13.374600 (+ 1401us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:13.374619 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:13.374660 (+ 41us) server_negotiation.cc:407] Connection header received
0806 18:44:13.375483 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:13.375508 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:13.375885 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:13.376252 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:13.376883 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:13.376888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:13.376903 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:13.376914 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:13.376968 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:13.376992 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:13.377124 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:13.377207 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 18:44:13.377385 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1227,"thread_start_us":145,"threads_started":1}
I20260806 18:44:13.378288 9385 log.cc:824] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:13.380407 9385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0: No bootstrap required, opened a new log
I20260806 18:44:13.381170 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:13.384343 9385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d6b014b3e4407396a2a8f35e6c54a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } }
I20260806 18:44:13.384567 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:13.383060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.199:41199 (local address 127.0.0.1:38599)
0806 18:44:13.383162 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:13.383166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:13.383332 (+ 166us) server_negotiation.cc:407] Connection header received
0806 18:44:13.383467 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:13.383470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:13.383504 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:13.383561 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:13.384206 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:13.384210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:13.384213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:13.384224 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:13.384252 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:13.384262 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:13.384331 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:13.384414 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 18:44:13.384455 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:44:13.384599 9385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:13.384666 9385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94d6b014b3e4407396a2a8f35e6c54a0, State: Initialized, Role: FOLLOWER
I20260806 18:44:13.385159 9385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d6b014b3e4407396a2a8f35e6c54a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } }
I20260806 18:44:13.385309 9385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:13.385378 9385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:13.385479 9385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:13.386349 9385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d6b014b3e4407396a2a8f35e6c54a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } }
I20260806 18:44:13.386677 9385 leader_election.cc:302] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94d6b014b3e4407396a2a8f35e6c54a0; no voters:
I20260806 18:44:13.386997 9385 leader_election.cc:288] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:13.387076 9400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:13.387303 9400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [term 1 LEADER]: Becoming Leader. State: Replica: 94d6b014b3e4407396a2a8f35e6c54a0, State: Running, Role: LEADER
I20260806 18:44:13.387645 9400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d6b014b3e4407396a2a8f35e6c54a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } }
I20260806 18:44:13.388296 9385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:13.388845 9403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94d6b014b3e4407396a2a8f35e6c54a0. Latest consensus state: current_term: 1 leader_uuid: "94d6b014b3e4407396a2a8f35e6c54a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d6b014b3e4407396a2a8f35e6c54a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } } }
I20260806 18:44:13.388983 9403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:13.389638 9401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94d6b014b3e4407396a2a8f35e6c54a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94d6b014b3e4407396a2a8f35e6c54a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38599 } } }
I20260806 18:44:13.389734 9401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0 [sys.catalog]: This master's current role is: LEADER
W20260806 18:44:13.393419 9421 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:13.393497 9421 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:13.393610 9409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:13.394536 9409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:13.399766 9409 catalog_manager.cc:1347] Generated new cluster ID: 53411386c78e418895eb1e98911b8113
I20260806 18:44:13.399828 9409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:13.437899 9409 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:13.438244 9409 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:13.447888 9409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94d6b014b3e4407396a2a8f35e6c54a0: Generated new TSK 0
I20260806 18:44:13.448616 9409 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:13.452618 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9287
2026-08-06T18:44:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1529 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:13 dist-test-slave-k5rr krb5kdc[9429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:13 dist-test-slave-k5rr krb5kdc[9429](info): set up 2 sockets
Aug 06 18:44:13 dist-test-slave-k5rr krb5kdc[9429](info): commencing operation
krb5kdc: starting...
W20260806 18:44:14.592275 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:14 dist-test-slave-k5rr krb5kdc[9429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:44:14.712062 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42447
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:45909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:42447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:14.822820 9460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:14.823081 9460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:14.823139 9460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:14.827092 9460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:14.827148 9460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:14.827174 9460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:14.827198 9460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:14.827227 9460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:14.832404 9460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:45909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:42447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:14.833478 9460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:14.834491 9460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:14.843014 9469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:14.842989 9465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:14.842994 9466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:14.843043 9460 server_base.cc:1034] running on GCE node
I20260806 18:44:14.843631 9460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:14.844712 9460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:14.845894 9460 hybrid_clock.cc:648] HybridClock initialized: now 1786041854845873 us; error 29 us; skew 500 ppm
Aug 06 18:44:14 dist-test-slave-k5rr krb5kdc[9429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041854, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:14.848665 9460 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:44:14.849663 9460 webserver.cc:460] Webserver started at http://127.8.241.254:38713/ using document root <none> and password file <none>
I20260806 18:44:14.850248 9460 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:14.850332 9460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:14.850557 9460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:14.852710 9460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0"
format_stamp: "Formatted at 2026-08-06 18:44:14 on dist-test-slave-k5rr"
server_key: "a2fd62eb73b15dca3906e2149eeb29e6"
server_key_iv: "55b76a5654a18fdecaa5d103624e5f6f"
server_key_version: "encryptionkey@0"
I20260806 18:44:14.853287 9460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0"
format_stamp: "Formatted at 2026-08-06 18:44:14 on dist-test-slave-k5rr"
server_key: "a2fd62eb73b15dca3906e2149eeb29e6"
server_key_iv: "55b76a5654a18fdecaa5d103624e5f6f"
server_key_version: "encryptionkey@0"
I20260806 18:44:14.857200 9460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:44:14.859652 9480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:14.860880 9460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:14.861037 9460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0"
format_stamp: "Formatted at 2026-08-06 18:44:14 on dist-test-slave-k5rr"
server_key: "a2fd62eb73b15dca3906e2149eeb29e6"
server_key_iv: "55b76a5654a18fdecaa5d103624e5f6f"
server_key_version: "encryptionkey@0"
I20260806 18:44:14.861148 9460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:14.875022 9460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:14.878311 9460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:14.878505 9460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:14.885896 9460 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:42447
I20260806 18:44:14.885926 9554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:42447 every 8 connection(s)
I20260806 18:44:14.886891 9460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:14.888329 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9460
I20260806 18:44:14.888460 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:14.888744 9159 external_mini_cluster.cc:1428] Setting key 88d748c1599b77e0132cc83eb4c103cc
I20260806 18:44:14.890012 9555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:14.896734 9555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0: Bootstrap starting.
Aug 06 18:44:14 dist-test-slave-k5rr krb5kdc[9429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:44:14.899010 9555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:14.899905 9555 log.cc:824] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:14.902146 9555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0: No bootstrap required, opened a new log
I20260806 18:44:14.903802 9563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:14.890186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59893 (local address 127.8.241.254:42447)
0806 18:44:14.890644 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:14.890668 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:14.890710 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:44:14.891588 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:14.891610 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:14.892029 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:14.892393 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:14.893167 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:14.894626 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:14.895510 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:14.896171 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:14.896328 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:14.898916 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:14.898922 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:14.898939 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:14.898972 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:14.900728 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:14.901276 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:14.901281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:14.901287 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:14.901435 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:14.901720 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:14.901723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:14.901725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:14.902213 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:14.902431 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:14.902665 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:44:14.902894 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":129,"threads_started":1}
I20260806 18:44:14.906425 9555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42447 } }
I20260806 18:44:14.906651 9555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:14.906689 9555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148445cefc1144d9b5cc4f1fa7d0c3d0, State: Initialized, Role: FOLLOWER
I20260806 18:44:14.906978 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:14.907147 9555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42447 } }
I20260806 18:44:14.907287 9555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:14.907349 9555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:14.907446 9555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:14.908344 9555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42447 } }
I20260806 18:44:14.908669 9555 leader_election.cc:302] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 148445cefc1144d9b5cc4f1fa7d0c3d0; no voters:
I20260806 18:44:14.908952 9555 leader_election.cc:288] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:14.909250 9575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:14.909466 9575 raft_consensus.cc:686] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [term 1 LEADER]: Becoming Leader. State: Replica: 148445cefc1144d9b5cc4f1fa7d0c3d0, State: Running, Role: LEADER
I20260806 18:44:14.909818 9575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42447 } }
I20260806 18:44:14.910364 9555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:14.911080 9575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 148445cefc1144d9b5cc4f1fa7d0c3d0. Latest consensus state: current_term: 1 leader_uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42447 } } }
I20260806 18:44:14.911214 9575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:14.913187 9577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148445cefc1144d9b5cc4f1fa7d0c3d0" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42447 } } }
I20260806 18:44:14.913288 9577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0 [sys.catalog]: This master's current role is: LEADER
W20260806 18:44:14.916127 9597 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:14.916226 9597 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:14.916320 9586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:14.917315 9586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:14.917557 9563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:14.909119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38973 (local address 127.8.241.254:42447)
0806 18:44:14.909220 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:14.909225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:14.909526 (+ 301us) server_negotiation.cc:407] Connection header received
0806 18:44:14.909711 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:14.909715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:14.909761 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:14.909839 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:14.910670 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:14.911470 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:14.912620 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:14.912944 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:14.913064 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:14.914479 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:14.914483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:14.914486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:14.914509 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:14.915791 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:14.916238 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:14.916241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:14.916242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:14.916326 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:14.916665 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:14.916669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:14.916670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:14.916840 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:14.916977 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:14.917266 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 18:44:14.917353 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 18:44:14.923187 9586 catalog_manager.cc:1347] Generated new cluster ID: a17c827c5e34484eaf1e70f5d117eaa6
I20260806 18:44:14.923249 9586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:14.929749 9586 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:14.930697 9586 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:14.938889 9586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 148445cefc1144d9b5cc4f1fa7d0c3d0: Generated new TSK 0
I20260806 18:44:14.939715 9586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:14.946172 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9460
W20260806 18:44:14.956759 9427 connection.cc:535] client connection to 127.8.241.254:42447 recv error: Network error: failed to read from TLS socket (remote: 127.8.241.254:42447): Connection reset by peer (error 104)
2026-08-06T18:44:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:15 dist-test-slave-k5rr krb5kdc[9605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:15 dist-test-slave-k5rr krb5kdc[9605](info): set up 2 sockets
Aug 06 18:44:15 dist-test-slave-k5rr krb5kdc[9605](info): commencing operation
krb5kdc: starting...
W20260806 18:44:16.078205 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:16 dist-test-slave-k5rr krb5kdc[9605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:44:16.203449 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36447
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:36447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:16.318295 9619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:16.318614 9619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:16.318673 9619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:16.322691 9619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:16.322760 9619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:16.322788 9619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:16.322808 9619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:16.322829 9619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:16.328128 9619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:36447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:16.329281 9619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:16.330368 9619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:16.339033 9625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:16.339069 9624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:16.339069 9627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:16.339123 9619 server_base.cc:1034] running on GCE node
I20260806 18:44:16.339805 9619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:16.340875 9619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:16.342046 9619 hybrid_clock.cc:648] HybridClock initialized: now 1786041856342027 us; error 36 us; skew 500 ppm
Aug 06 18:44:16 dist-test-slave-k5rr krb5kdc[9605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041856, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:16.348862 9619 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:44:16.349871 9619 webserver.cc:460] Webserver started at http://127.8.241.254:35385/ using document root <none> and password file <none>
I20260806 18:44:16.350452 9619 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:16.350521 9619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:16.350739 9619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:16.352880 9619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "63b856b919124cef8f495e131556c919"
format_stamp: "Formatted at 2026-08-06 18:44:16 on dist-test-slave-k5rr"
server_key: "2a01fbeb69e5ed112f36036640fcb844"
server_key_iv: "65800473e7fa7a028a30c34499b2ea4e"
server_key_version: "encryptionkey@0"
I20260806 18:44:16.353484 9619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "63b856b919124cef8f495e131556c919"
format_stamp: "Formatted at 2026-08-06 18:44:16 on dist-test-slave-k5rr"
server_key: "2a01fbeb69e5ed112f36036640fcb844"
server_key_iv: "65800473e7fa7a028a30c34499b2ea4e"
server_key_version: "encryptionkey@0"
I20260806 18:44:16.357266 9619 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:44:16.359728 9639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:16.360996 9619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:16.361148 9619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "63b856b919124cef8f495e131556c919"
format_stamp: "Formatted at 2026-08-06 18:44:16 on dist-test-slave-k5rr"
server_key: "2a01fbeb69e5ed112f36036640fcb844"
server_key_iv: "65800473e7fa7a028a30c34499b2ea4e"
server_key_version: "encryptionkey@0"
I20260806 18:44:16.361243 9619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:16.373708 9619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:16.376384 9619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:16.376559 9619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:16.384297 9619 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:36447
I20260806 18:44:16.384341 9717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:36447 every 8 connection(s)
I20260806 18:44:16.385425 9619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:16.388386 9718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:16.390705 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9619
I20260806 18:44:16.390859 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:16.391189 9159 external_mini_cluster.cc:1428] Setting key 002bd1c143cfc73b051c294c6ad6926e
I20260806 18:44:16.395099 9718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919: Bootstrap starting.
I20260806 18:44:16.397503 9718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:16.398280 9718 log.cc:824] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:16.400478 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:16.392771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58207 (local address 127.8.241.254:36447)
0806 18:44:16.393276 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:16.393291 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:16.393334 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:44:16.394166 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:16.394186 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:16.394536 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:16.394858 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:16.395287 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:16.396265 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:16.397177 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:16.397668 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:16.397825 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:16.398700 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:16.398708 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:16.398726 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:16.399250 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:16.399308 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:16.399332 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:16.399447 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:16.399636 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 18:44:16.399782 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":161,"threads_started":1}
I20260806 18:44:16.400573 9718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919: No bootstrap required, opened a new log
I20260806 18:44:16.403420 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:16.404374 9718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b856b919124cef8f495e131556c919" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36447 } }
I20260806 18:44:16.404656 9718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:16.404770 9718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63b856b919124cef8f495e131556c919, State: Initialized, Role: FOLLOWER
I20260806 18:44:16.405272 9718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b856b919124cef8f495e131556c919" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36447 } }
I20260806 18:44:16.405457 9718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:16.405540 9718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:16.405654 9718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:16.406545 9718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b856b919124cef8f495e131556c919" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36447 } }
I20260806 18:44:16.406869 9718 leader_election.cc:302] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63b856b919124cef8f495e131556c919; no voters:
I20260806 18:44:16.407136 9718 leader_election.cc:288] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:16.407274 9731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:16.407487 9731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [term 1 LEADER]: Becoming Leader. State: Replica: 63b856b919124cef8f495e131556c919, State: Running, Role: LEADER
I20260806 18:44:16.407802 9731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b856b919124cef8f495e131556c919" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36447 } }
I20260806 18:44:16.408522 9718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:16.408983 9733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63b856b919124cef8f495e131556c919. Latest consensus state: current_term: 1 leader_uuid: "63b856b919124cef8f495e131556c919" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b856b919124cef8f495e131556c919" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36447 } } }
I20260806 18:44:16.409143 9733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:16.409510 9734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63b856b919124cef8f495e131556c919" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63b856b919124cef8f495e131556c919" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36447 } } }
I20260806 18:44:16.409610 9734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:16.412071 9722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:16.405363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59337 (local address 127.8.241.254:36447)
0806 18:44:16.405462 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:16.405467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:16.407514 (+ 2047us) server_negotiation.cc:407] Connection header received
0806 18:44:16.407707 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:16.407711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:16.407759 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:16.407814 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:16.408289 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:16.409116 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:16.410251 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:16.410546 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:16.410641 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:16.411591 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:16.411595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:16.411598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:16.411610 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:16.411637 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:16.411650 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:16.411733 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:16.411913 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 18:44:16.411949 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 18:44:16.413875 9750 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:16.413944 9750 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:16.414067 9738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:16.415028 9738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:16.420794 9738 catalog_manager.cc:1347] Generated new cluster ID: 285ed3f46e864af9bb8ecb5dc0542d91
I20260806 18:44:16.420846 9738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:16.429692 9738 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:16.430948 9738 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:16.444623 9738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63b856b919124cef8f495e131556c919: Generated new TSK 0
I20260806 18:44:16.445529 9738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:16.457000 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9619
W20260806 18:44:16.469668 9601 connection.cc:535] client connection to 127.8.241.254:36447 recv error: Network error: failed to read from TLS socket (remote: 127.8.241.254:36447): Connection reset by peer (error 104)
2026-08-06T18:44:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:16 dist-test-slave-k5rr krb5kdc[9757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:16 dist-test-slave-k5rr krb5kdc[9757](info): set up 2 sockets
Aug 06 18:44:16 dist-test-slave-k5rr krb5kdc[9757](info): commencing operation
krb5kdc: starting...
W20260806 18:44:17.597518 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:17 dist-test-slave-k5rr krb5kdc[9757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:44:17.720930 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:35187
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:32839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:35187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:17.833149 9772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:17.833425 9772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:17.833467 9772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:17.837535 9772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:17.837597 9772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:17.837618 9772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:17.837633 9772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:17.837648 9772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:17.842866 9772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:32839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:35187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:35187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:17.843840 9772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:17.844805 9772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:17.853279 9784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:17.853313 9780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:17.853403 9781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:17.853520 9772 server_base.cc:1034] running on GCE node
I20260806 18:44:17.854198 9772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:17.855331 9772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:17.856495 9772 hybrid_clock.cc:648] HybridClock initialized: now 1786041857856479 us; error 31 us; skew 500 ppm
Aug 06 18:44:17 dist-test-slave-k5rr krb5kdc[9757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041857, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:17.859180 9772 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:44:17.860039 9772 webserver.cc:460] Webserver started at http://127.8.241.254:37187/ using document root <none> and password file <none>
I20260806 18:44:17.860569 9772 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:17.860626 9772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:17.860795 9772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:17.862869 9772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "f6f3099d725440729f7c8759f2528d2e"
format_stamp: "Formatted at 2026-08-06 18:44:17 on dist-test-slave-k5rr"
server_key: "c4d1a3ef91b073e3d376a362e5cb8a12"
server_key_iv: "7baa5039c1714f2ea4f2e64bd8bc736b"
server_key_version: "encryptionkey@0"
I20260806 18:44:17.863389 9772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "f6f3099d725440729f7c8759f2528d2e"
format_stamp: "Formatted at 2026-08-06 18:44:17 on dist-test-slave-k5rr"
server_key: "c4d1a3ef91b073e3d376a362e5cb8a12"
server_key_iv: "7baa5039c1714f2ea4f2e64bd8bc736b"
server_key_version: "encryptionkey@0"
I20260806 18:44:17.867086 9772 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:44:17.869463 9791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:17.870684 9772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:17.870800 9772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "f6f3099d725440729f7c8759f2528d2e"
format_stamp: "Formatted at 2026-08-06 18:44:17 on dist-test-slave-k5rr"
server_key: "c4d1a3ef91b073e3d376a362e5cb8a12"
server_key_iv: "7baa5039c1714f2ea4f2e64bd8bc736b"
server_key_version: "encryptionkey@0"
I20260806 18:44:17.870900 9772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:17.875687 9772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:17.878660 9772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:17.878834 9772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:17.885890 9772 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:35187
I20260806 18:44:17.885911 9868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:35187 every 8 connection(s)
I20260806 18:44:17.886901 9772 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:17.887538 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9772
I20260806 18:44:17.887671 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:17.887918 9159 external_mini_cluster.cc:1428] Setting key eefb89c5bb9a59c9f95c8948cfe1a038
I20260806 18:44:17.890106 9869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:17.893026 9873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:17.889303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52737 (local address 127.8.241.254:35187)
0806 18:44:17.889748 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:17.889763 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:17.889797 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:44:17.890479 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:17.890499 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:17.890783 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:17.891100 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:17.891871 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:17.891875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:17.891887 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:17.891898 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:17.891949 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:17.891968 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:17.892094 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:17.892186 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 18:44:17.892335 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":149,"threads_started":1}
I20260806 18:44:17.895635 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:17.896103 9869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e: Bootstrap starting.
I20260806 18:44:17.898564 9869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:17.899075 9873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:17.897557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60523 (local address 127.8.241.254:35187)
0806 18:44:17.897654 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:17.897658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:17.897854 (+ 196us) server_negotiation.cc:407] Connection header received
0806 18:44:17.897963 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:17.897966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:17.897999 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:17.898060 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:17.898699 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:17.898703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:17.898706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:17.898717 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:17.898743 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:17.898753 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:17.898819 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:17.898928 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 18:44:17.898965 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 18:44:17.899432 9869 log.cc:824] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:17.901448 9869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e: No bootstrap required, opened a new log
I20260806 18:44:17.904984 9869 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6f3099d725440729f7c8759f2528d2e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35187 } }
I20260806 18:44:17.905251 9869 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:17.905313 9869 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6f3099d725440729f7c8759f2528d2e, State: Initialized, Role: FOLLOWER
I20260806 18:44:17.905712 9869 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6f3099d725440729f7c8759f2528d2e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35187 } }
I20260806 18:44:17.905844 9869 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:17.905910 9869 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:17.906006 9869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:17.906814 9869 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6f3099d725440729f7c8759f2528d2e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35187 } }
I20260806 18:44:17.907135 9869 leader_election.cc:302] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6f3099d725440729f7c8759f2528d2e; no voters:
I20260806 18:44:17.907435 9869 leader_election.cc:288] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:17.907526 9882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:17.907699 9882 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [term 1 LEADER]: Becoming Leader. State: Replica: f6f3099d725440729f7c8759f2528d2e, State: Running, Role: LEADER
I20260806 18:44:17.907992 9882 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6f3099d725440729f7c8759f2528d2e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35187 } }
I20260806 18:44:17.908715 9869 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:17.909722 9884 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6f3099d725440729f7c8759f2528d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6f3099d725440729f7c8759f2528d2e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35187 } } }
I20260806 18:44:17.909853 9884 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:17.910130 9886 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6f3099d725440729f7c8759f2528d2e. Latest consensus state: current_term: 1 leader_uuid: "f6f3099d725440729f7c8759f2528d2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6f3099d725440729f7c8759f2528d2e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 35187 } } }
I20260806 18:44:17.910219 9886 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:17.910398 9896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:17.913164 9896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:17.919024 9896 catalog_manager.cc:1347] Generated new cluster ID: 1a26865af3d84bd3b8309bbc3e0c338c
I20260806 18:44:17.919109 9896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:17.951838 9896 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:17.952193 9896 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:17.958937 9896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6f3099d725440729f7c8759f2528d2e: Generated new TSK 0
I20260806 18:44:17.959602 9896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:17.962541 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9772
2026-08-06T18:44:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1504 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:18 dist-test-slave-k5rr krb5kdc[9916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:18 dist-test-slave-k5rr krb5kdc[9916](info): set up 2 sockets
Aug 06 18:44:18 dist-test-slave-k5rr krb5kdc[9916](info): commencing operation
krb5kdc: starting...
W20260806 18:44:19.094374 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:19 dist-test-slave-k5rr krb5kdc[9916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:44:19.214152 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34489
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:34903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:34489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:19.325938 9934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:19.326190 9934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:19.326229 9934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:19.330065 9934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:19.330117 9934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:19.330134 9934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:19.330149 9934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:19.330163 9934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:19.335270 9934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:34903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:34489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:34489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:19.336289 9934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:19.337361 9934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:19.345602 9944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:19.345609 9940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:19.345788 9934 server_base.cc:1034] running on GCE node
W20260806 18:44:19.345602 9939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:19.346453 9934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:19.347517 9934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:19.348675 9934 hybrid_clock.cc:648] HybridClock initialized: now 1786041859348662 us; error 27 us; skew 500 ppm
Aug 06 18:44:19 dist-test-slave-k5rr krb5kdc[9916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041859, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:19.351233 9934 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:44:19.352090 9934 webserver.cc:460] Webserver started at http://127.8.241.254:42981/ using document root <none> and password file <none>
I20260806 18:44:19.352608 9934 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:19.352671 9934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:19.352871 9934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:19.354951 9934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "d7504bd922d9420a8550d8e3d606930e"
format_stamp: "Formatted at 2026-08-06 18:44:19 on dist-test-slave-k5rr"
server_key: "78e09f699d0aabf3165dbd4f904a8a5b"
server_key_iv: "6624ea92adca68afb186ed8eb0679cc5"
server_key_version: "encryptionkey@0"
I20260806 18:44:19.355511 9934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "d7504bd922d9420a8550d8e3d606930e"
format_stamp: "Formatted at 2026-08-06 18:44:19 on dist-test-slave-k5rr"
server_key: "78e09f699d0aabf3165dbd4f904a8a5b"
server_key_iv: "6624ea92adca68afb186ed8eb0679cc5"
server_key_version: "encryptionkey@0"
I20260806 18:44:19.359073 9934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 18:44:19.361332 9952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:19.362497 9934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:19.362612 9934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "d7504bd922d9420a8550d8e3d606930e"
format_stamp: "Formatted at 2026-08-06 18:44:19 on dist-test-slave-k5rr"
server_key: "78e09f699d0aabf3165dbd4f904a8a5b"
server_key_iv: "6624ea92adca68afb186ed8eb0679cc5"
server_key_version: "encryptionkey@0"
I20260806 18:44:19.362704 9934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:19.376621 9934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:19.379446 9934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:19.379642 9934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:19.386711 9934 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:34489
I20260806 18:44:19.386729 10027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:34489 every 8 connection(s)
I20260806 18:44:19.387660 9934 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:19.390076 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 9934
I20260806 18:44:19.390208 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:19.390517 9159 external_mini_cluster.cc:1428] Setting key 52cab543b72081d93c779765ba60a071
I20260806 18:44:19.390733 10028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:19.398284 10028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e: Bootstrap starting.
Aug 06 18:44:19 dist-test-slave-k5rr krb5kdc[9916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:44:19.400800 10028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:19.401633 10028 log.cc:824] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:19.403941 10028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e: No bootstrap required, opened a new log
I20260806 18:44:19.405716 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:19.391890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38907 (local address 127.8.241.254:34489)
0806 18:44:19.392367 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:19.392384 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:19.392422 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:44:19.393277 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:19.393312 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:19.393690 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:19.394017 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:19.395437 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:19.396922 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:19.397796 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:19.398471 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:19.398624 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:19.401232 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:19.401237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:19.401251 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:19.401279 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:19.402992 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:19.403448 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:19.403452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:19.403457 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:19.403556 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:19.403831 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:19.403833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:19.403834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:19.404234 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:19.404382 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:19.404660 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 18:44:19.404942 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":136,"threads_started":1}
I20260806 18:44:19.407627 10028 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7504bd922d9420a8550d8e3d606930e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34489 } }
I20260806 18:44:19.407980 10028 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:19.408061 10028 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7504bd922d9420a8550d8e3d606930e, State: Initialized, Role: FOLLOWER
I20260806 18:44:19.408569 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:19.408545 10028 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7504bd922d9420a8550d8e3d606930e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34489 } }
I20260806 18:44:19.408716 10028 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:19.408780 10028 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:19.408870 10028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:19.410020 10028 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7504bd922d9420a8550d8e3d606930e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34489 } }
I20260806 18:44:19.410406 10028 leader_election.cc:302] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7504bd922d9420a8550d8e3d606930e; no voters:
I20260806 18:44:19.410733 10028 leader_election.cc:288] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:19.410840 10038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:19.411026 10038 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [term 1 LEADER]: Becoming Leader. State: Replica: d7504bd922d9420a8550d8e3d606930e, State: Running, Role: LEADER
I20260806 18:44:19.411324 10038 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7504bd922d9420a8550d8e3d606930e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34489 } }
I20260806 18:44:19.412041 10028 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:19.412614 10042 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7504bd922d9420a8550d8e3d606930e. Latest consensus state: current_term: 1 leader_uuid: "d7504bd922d9420a8550d8e3d606930e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7504bd922d9420a8550d8e3d606930e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34489 } } }
I20260806 18:44:19.412765 10042 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:19.413182 10052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:19.413475 10040 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7504bd922d9420a8550d8e3d606930e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7504bd922d9420a8550d8e3d606930e" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 34489 } } }
I20260806 18:44:19.413574 10040 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:19.416759 10052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:19.420357 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:19.410936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54079 (local address 127.8.241.254:34489)
0806 18:44:19.411199 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:19.411205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:19.411430 (+ 225us) server_negotiation.cc:407] Connection header received
0806 18:44:19.411605 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:19.411609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:19.411661 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:19.411742 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:19.412619 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:19.413407 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:19.414623 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:19.414958 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:19.415051 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:19.416707 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:19.416712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:19.416715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:19.416742 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:19.418537 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:19.419034 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:19.419038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:19.419039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:19.419139 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:19.419482 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:19.419486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:19.419488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:19.419678 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:19.419815 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:19.420059 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 18:44:19.420179 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260806 18:44:19.423688 10052 catalog_manager.cc:1347] Generated new cluster ID: 9fd9457a2f044985b8d4ef89151e00f7
I20260806 18:44:19.423759 10052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:19.430037 10052 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:19.430908 10052 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:19.435318 10052 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7504bd922d9420a8550d8e3d606930e: Generated new TSK 0
I20260806 18:44:19.436064 10052 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:19.440595 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 9934
2026-08-06T18:44:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1483 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:19 dist-test-slave-k5rr krb5kdc[10068](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:19 dist-test-slave-k5rr krb5kdc[10068](info): set up 2 sockets
Aug 06 18:44:19 dist-test-slave-k5rr krb5kdc[10068](info): commencing operation
krb5kdc: starting...
W20260806 18:44:20.588037 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:20 dist-test-slave-k5rr krb5kdc[10068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:44:20.715947 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42707
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:39225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:42707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:20.831144 10086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:20.831411 10086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:20.831470 10086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:20.835450 10086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:20.835525 10086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:20.835541 10086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:20.835562 10086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:20.835577 10086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:20.841001 10086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:39225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:42707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:42707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:20.842082 10086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:20.843091 10086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:20.851858 10094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:20.852003 10086 server_base.cc:1034] running on GCE node
W20260806 18:44:20.851809 10093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:20.851848 10096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:20.852617 10086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:20.853814 10086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:20.854969 10086 hybrid_clock.cc:648] HybridClock initialized: now 1786041860854945 us; error 37 us; skew 500 ppm
Aug 06 18:44:20 dist-test-slave-k5rr krb5kdc[10068](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041860, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:20.857656 10086 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:44:20.858609 10086 webserver.cc:460] Webserver started at http://127.8.241.254:37447/ using document root <none> and password file <none>
I20260806 18:44:20.859177 10086 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:20.859246 10086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:20.859467 10086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:20.861636 10086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "38235b3499fd4b38bab6d09e6761689c"
format_stamp: "Formatted at 2026-08-06 18:44:20 on dist-test-slave-k5rr"
server_key: "f80ddb3df170ef6f27882bd3b4988664"
server_key_iv: "a14681f9ca03f7ee48cf9d676d972272"
server_key_version: "encryptionkey@0"
I20260806 18:44:20.862185 10086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "38235b3499fd4b38bab6d09e6761689c"
format_stamp: "Formatted at 2026-08-06 18:44:20 on dist-test-slave-k5rr"
server_key: "f80ddb3df170ef6f27882bd3b4988664"
server_key_iv: "a14681f9ca03f7ee48cf9d676d972272"
server_key_version: "encryptionkey@0"
I20260806 18:44:20.865818 10086 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:44:20.868114 10103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:20.869313 10086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:20.869439 10086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "38235b3499fd4b38bab6d09e6761689c"
format_stamp: "Formatted at 2026-08-06 18:44:20 on dist-test-slave-k5rr"
server_key: "f80ddb3df170ef6f27882bd3b4988664"
server_key_iv: "a14681f9ca03f7ee48cf9d676d972272"
server_key_version: "encryptionkey@0"
I20260806 18:44:20.869531 10086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:20.882953 10086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:20.885874 10086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:20.886063 10086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:20.892933 10086 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:42707
I20260806 18:44:20.892949 10190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:42707 every 8 connection(s)
I20260806 18:44:20.893877 10086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:20.896624 10191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:20.902266 10191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c: Bootstrap starting.
I20260806 18:44:20.902532 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10086
I20260806 18:44:20.902669 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:20.902989 9159 external_mini_cluster.cc:1428] Setting key d227f117db5ac5450da201f99eb2ac4e
I20260806 18:44:20.905033 10191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:20.905786 10191 log.cc:824] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:20.907712 10191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c: No bootstrap required, opened a new log
I20260806 18:44:20.911783 10191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38235b3499fd4b38bab6d09e6761689c" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42707 } }
I20260806 18:44:20.912000 10191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:20.912050 10191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38235b3499fd4b38bab6d09e6761689c, State: Initialized, Role: FOLLOWER
I20260806 18:44:20.912189 10194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:20.904344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46319 (local address 127.8.241.254:42707)
0806 18:44:20.904795 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:20.904811 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:20.904848 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:44:20.905576 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:20.905597 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:20.905959 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:20.906307 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:20.906697 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:20.907655 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:20.908873 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:20.909411 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:20.909538 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:20.910458 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:20.910463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:20.910477 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:20.910872 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:20.910929 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:20.910953 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:20.911030 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:20.911173 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:44:20.911388 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":138,"threads_started":1}
I20260806 18:44:20.912529 10191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38235b3499fd4b38bab6d09e6761689c" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42707 } }
I20260806 18:44:20.912693 10191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:20.912758 10191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:20.912850 10191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:20.913815 10191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38235b3499fd4b38bab6d09e6761689c" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42707 } }
I20260806 18:44:20.914141 10191 leader_election.cc:302] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38235b3499fd4b38bab6d09e6761689c; no voters:
I20260806 18:44:20.914482 10191 leader_election.cc:288] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:20.914557 10196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:20.914733 10196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [term 1 LEADER]: Becoming Leader. State: Replica: 38235b3499fd4b38bab6d09e6761689c, State: Running, Role: LEADER
I20260806 18:44:20.915081 10196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38235b3499fd4b38bab6d09e6761689c" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42707 } }
I20260806 18:44:20.915175 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:20.915777 10191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:20.917551 10197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38235b3499fd4b38bab6d09e6761689c. Latest consensus state: current_term: 1 leader_uuid: "38235b3499fd4b38bab6d09e6761689c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38235b3499fd4b38bab6d09e6761689c" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42707 } } }
I20260806 18:44:20.917800 10198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38235b3499fd4b38bab6d09e6761689c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38235b3499fd4b38bab6d09e6761689c" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 42707 } } }
I20260806 18:44:20.917891 10197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:20.918001 10198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c [sys.catalog]: This master's current role is: LEADER
W20260806 18:44:20.920480 10222 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:20.920545 10222 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:20.920624 10224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:20.921599 10224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:20.925061 10194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:20.917617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40915 (local address 127.8.241.254:42707)
0806 18:44:20.918200 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:20.918204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:20.920972 (+ 2768us) server_negotiation.cc:407] Connection header received
0806 18:44:20.921149 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:20.921154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:20.921205 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:20.921258 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:20.921690 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:20.922512 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:20.923584 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:20.923805 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:20.923902 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:20.924648 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:20.924651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:20.924653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:20.924664 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:20.924688 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:20.924698 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:20.924757 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:20.924846 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 18:44:20.924904 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515}
I20260806 18:44:20.927881 10224 catalog_manager.cc:1347] Generated new cluster ID: 4b10e79fc29b47c0b54a75437a376839
I20260806 18:44:20.927938 10224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:20.939807 10224 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:20.940820 10224 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:20.948334 10224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38235b3499fd4b38bab6d09e6761689c: Generated new TSK 0
I20260806 18:44:20.949070 10224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:20.953145 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10086
2026-08-06T18:44:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:21 dist-test-slave-k5rr krb5kdc[10229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:21 dist-test-slave-k5rr krb5kdc[10229](info): set up 2 sockets
Aug 06 18:44:21 dist-test-slave-k5rr krb5kdc[10229](info): commencing operation
krb5kdc: starting...
W20260806 18:44:22.086781 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:22 dist-test-slave-k5rr krb5kdc[10229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:44:22.209658 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:44299
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:44299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:22.326017 10250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:22.326278 10250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:22.326337 10250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:22.330305 10250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:22.330366 10250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:22.330394 10250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:22.330417 10250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:22.330447 10250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:22.335705 10250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:44299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:44299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:22.336786 10250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:22.337873 10250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:22.346364 10258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:22.346380 10261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:22.346380 10256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:22.346510 10250 server_base.cc:1034] running on GCE node
I20260806 18:44:22.347190 10250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:22.348289 10250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:22.349449 10250 hybrid_clock.cc:648] HybridClock initialized: now 1786041862349432 us; error 27 us; skew 500 ppm
Aug 06 18:44:22 dist-test-slave-k5rr krb5kdc[10229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041862, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:22.355227 10250 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:44:22.356141 10250 webserver.cc:460] Webserver started at http://127.8.241.254:40139/ using document root <none> and password file <none>
I20260806 18:44:22.356712 10250 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:22.356781 10250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:22.357029 10250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:22.359370 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "53cfab0d289b45fb844b0bdd86f7c340"
format_stamp: "Formatted at 2026-08-06 18:44:22 on dist-test-slave-k5rr"
server_key: "ff428c15031c491a0480728a9845ef40"
server_key_iv: "e000d95298885a69f258d25f7cf208fc"
server_key_version: "encryptionkey@0"
I20260806 18:44:22.359912 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "53cfab0d289b45fb844b0bdd86f7c340"
format_stamp: "Formatted at 2026-08-06 18:44:22 on dist-test-slave-k5rr"
server_key: "ff428c15031c491a0480728a9845ef40"
server_key_iv: "e000d95298885a69f258d25f7cf208fc"
server_key_version: "encryptionkey@0"
I20260806 18:44:22.363682 10250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:44:22.366127 10269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:22.367420 10250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:44:22.367542 10250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "53cfab0d289b45fb844b0bdd86f7c340"
format_stamp: "Formatted at 2026-08-06 18:44:22 on dist-test-slave-k5rr"
server_key: "ff428c15031c491a0480728a9845ef40"
server_key_iv: "e000d95298885a69f258d25f7cf208fc"
server_key_version: "encryptionkey@0"
I20260806 18:44:22.367637 10250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:22.372117 10250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:22.375164 10250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:22.375329 10250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:22.382206 10250 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:44299
I20260806 18:44:22.382215 10355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:44299 every 8 connection(s)
I20260806 18:44:22.383157 10250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:22.386078 10356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:22.387660 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10250
I20260806 18:44:22.387786 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:22.388068 9159 external_mini_cluster.cc:1428] Setting key d568a63f293663302eaa58a0b26fc56a
I20260806 18:44:22.392191 10356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340: Bootstrap starting.
I20260806 18:44:22.393178 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:22.389471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39985 (local address 127.8.241.254:44299)
0806 18:44:22.389896 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:22.389912 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:22.389949 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:44:22.390597 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:22.390614 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:22.390898 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:22.391194 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:22.391971 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:22.391976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:22.391988 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:22.391998 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:22.392051 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:22.392070 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:22.392194 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:22.392269 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:44:22.392442 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":129,"threads_started":1}
I20260806 18:44:22.394680 10356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:22.395521 10356 log.cc:824] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:22.395982 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:22.397830 10356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340: No bootstrap required, opened a new log
I20260806 18:44:22.399701 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:22.398201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60731 (local address 127.8.241.254:44299)
0806 18:44:22.398318 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:22.398322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:22.398503 (+ 181us) server_negotiation.cc:407] Connection header received
0806 18:44:22.398632 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:22.398634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:22.398662 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:22.398722 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:22.399385 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:22.399388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:22.399390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:22.399400 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:22.399423 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:22.399433 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:22.399492 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:22.399572 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 18:44:22.399602 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:44:22.402053 10356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53cfab0d289b45fb844b0bdd86f7c340" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44299 } }
I20260806 18:44:22.402361 10356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:22.402428 10356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53cfab0d289b45fb844b0bdd86f7c340, State: Initialized, Role: FOLLOWER
I20260806 18:44:22.402930 10356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53cfab0d289b45fb844b0bdd86f7c340" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44299 } }
I20260806 18:44:22.403160 10356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:22.403239 10356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:22.403342 10356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:22.404577 10356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53cfab0d289b45fb844b0bdd86f7c340" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44299 } }
I20260806 18:44:22.404940 10356 leader_election.cc:302] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53cfab0d289b45fb844b0bdd86f7c340; no voters:
I20260806 18:44:22.405294 10356 leader_election.cc:288] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:22.405390 10370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:22.405613 10370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [term 1 LEADER]: Becoming Leader. State: Replica: 53cfab0d289b45fb844b0bdd86f7c340, State: Running, Role: LEADER
I20260806 18:44:22.405954 10370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53cfab0d289b45fb844b0bdd86f7c340" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44299 } }
I20260806 18:44:22.406483 10356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:22.407557 10373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53cfab0d289b45fb844b0bdd86f7c340. Latest consensus state: current_term: 1 leader_uuid: "53cfab0d289b45fb844b0bdd86f7c340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53cfab0d289b45fb844b0bdd86f7c340" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44299 } } }
I20260806 18:44:22.407684 10373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:22.407953 10372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53cfab0d289b45fb844b0bdd86f7c340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53cfab0d289b45fb844b0bdd86f7c340" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 44299 } } }
I20260806 18:44:22.408041 10372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:22.408062 10390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:22.411373 10390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:22.416798 10390 catalog_manager.cc:1347] Generated new cluster ID: 23e9ac86d700420089a9c750adc982d5
I20260806 18:44:22.416872 10390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:22.430251 10390 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:22.430573 10390 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:22.438381 10390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53cfab0d289b45fb844b0bdd86f7c340: Generated new TSK 0
I20260806 18:44:22.439237 10390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:22.442782 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10250
2026-08-06T18:44:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1488 ms)
[----------] 18 tests from AuthTokenIssuingTest (27152 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:22 dist-test-slave-k5rr krb5kdc[10403](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:22 dist-test-slave-k5rr krb5kdc[10403](info): set up 2 sockets
Aug 06 18:44:22 dist-test-slave-k5rr krb5kdc[10403](info): commencing operation
krb5kdc: starting...
W20260806 18:44:23.595614 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:23 dist-test-slave-k5rr krb5kdc[10403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:44:23.718134 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36407
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:46271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:36407,127.8.241.253:37133,127.8.241.252:34377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:23.833846 10422 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:44:23.834197 10422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:23.834233 10422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:23.834295 10422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:23.838305 10422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:23.838368 10422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:23.838387 10422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:23.838402 10422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:23.838415 10422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:23.843859 10422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:46271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:36407,127.8.241.253:37133,127.8.241.252:34377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:36407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:23.845072 10422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:23.846212 10422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:23.855054 10431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:23.855043 10434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:23.855065 10430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:23.855477 10422 server_base.cc:1034] running on GCE node
I20260806 18:44:23.856011 10422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:23.857180 10422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:23.858332 10422 hybrid_clock.cc:648] HybridClock initialized: now 1786041863858306 us; error 41 us; skew 500 ppm
Aug 06 18:44:23 dist-test-slave-k5rr krb5kdc[10403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041863, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:23.861119 10422 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:44:23.862100 10422 webserver.cc:460] Webserver started at http://127.8.241.254:46143/ using document root <none> and password file <none>
I20260806 18:44:23.862679 10422 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:23.862756 10422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:23.862993 10422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:23.865338 10422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "69dd35d4cd53422c8b44e7b1f35489f4"
format_stamp: "Formatted at 2026-08-06 18:44:23 on dist-test-slave-k5rr"
server_key: "b89566cc679d0062912afb46471b89df"
server_key_iv: "f423992684b5c8d0da91e675ec2b5e6d"
server_key_version: "encryptionkey@0"
I20260806 18:44:23.865921 10422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "69dd35d4cd53422c8b44e7b1f35489f4"
format_stamp: "Formatted at 2026-08-06 18:44:23 on dist-test-slave-k5rr"
server_key: "b89566cc679d0062912afb46471b89df"
server_key_iv: "f423992684b5c8d0da91e675ec2b5e6d"
server_key_version: "encryptionkey@0"
I20260806 18:44:23.869812 10422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:44:23.872390 10443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:23.873697 10422 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:44:23.873852 10422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "69dd35d4cd53422c8b44e7b1f35489f4"
format_stamp: "Formatted at 2026-08-06 18:44:23 on dist-test-slave-k5rr"
server_key: "b89566cc679d0062912afb46471b89df"
server_key_iv: "f423992684b5c8d0da91e675ec2b5e6d"
server_key_version: "encryptionkey@0"
I20260806 18:44:23.873953 10422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:23.887666 10422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:23.890589 10422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:23.890789 10422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:23.898030 10422 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:36407
I20260806 18:44:23.898295 10532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:36407 every 8 connection(s)
I20260806 18:44:23.899030 10422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:23.901919 10534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:23.905468 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10422
I20260806 18:44:23.905622 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:23.905893 9159 external_mini_cluster.cc:1428] Setting key 92bf4ce64db72a48bb00d16c6d31a3f5
I20260806 18:44:23.906204 10534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.241.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.253@KRBTEST.COM" created.
Aug 06 18:44:23 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041863, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:44:23.922906 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:23.908854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46476 (local address 127.8.241.254:36407)
0806 18:44:23.909481 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:23.909500 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:23.909635 (+ 135us) server_negotiation.cc:407] Connection header received
0806 18:44:23.910886 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:23.910891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:23.911088 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:23.911269 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:23.912216 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:23.913159 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:23.914121 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:23.914531 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:23.914683 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:23.918089 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:23.918096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:23.918109 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:23.918132 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:23.919701 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:23.920153 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:23.920159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:23.920161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:23.920269 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:23.920553 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:23.920556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:23.920558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:23.920957 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:23.921137 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:23.921513 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 18:44:23.921777 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"spinlock_wait_cycles":34176,"thread_start_us":220,"threads_started":1}
I20260806 18:44:23.922906 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:23.908847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36407 (local address 127.0.0.1:46476)
0806 18:44:23.909464 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:23.909520 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:23.910461 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:23.911304 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:23.911310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:23.911810 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:23.912064 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:23.912081 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:23.913274 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:23.913277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:23.914007 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:23.914017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:23.914687 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:23.914692 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:23.914813 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:23.915351 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:23.915369 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:23.917941 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:23.919811 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:23.919821 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:23.919833 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:23.920053 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:23.920354 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:23.920358 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:23.920360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:23.920471 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:23.921105 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:23.921111 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:23.921272 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 18:44:23.921431 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 18:44:23.921846 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":94,"thread_start_us":189,"threads_started":1}
I20260806 18:44:23.925768 10534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:23.926543 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:23.926270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:37133 (local address 127.0.0.1:48826)
0806 18:44:23.926397 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:23.926472 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 18:44:23.926745 10450 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
I20260806 18:44:23.927690 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:23.927437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:37133 (local address 127.0.0.1:48834)
0806 18:44:23.927563 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:23.927619 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 18:44:23.927876 10534 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.253:37133: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab.
Entry for principal kudu/127.8.241.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab.
I20260806 18:44:23.933504 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.253:37133
--webserver_interface=127.8.241.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab
--principal=kudu/127.8.241.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:46271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:36407,127.8.241.253:37133,127.8.241.252:34377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:44:23.977092 10534 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } attempt: 1
I20260806 18:44:23.978082 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:23.977718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:37133 (local address 127.0.0.1:48836)
0806 18:44:23.977904 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:23.977984 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 18:44:23.978828 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:23.978582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:37133 (local address 127.0.0.1:48838)
0806 18:44:23.978681 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:23.978734 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 18:44:23.978961 10534 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.253:37133: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
I20260806 18:44:24.047153 10534 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } attempt: 2
I20260806 18:44:24.048086 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.047751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:37133 (local address 127.0.0.1:48844)
0806 18:44:24.047920 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.047996 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 18:44:24.048849 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.048596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:37133 (local address 127.0.0.1:48848)
0806 18:44:24.048690 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.048781 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 18:44:24.048970 10534 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.253:37133: Network error: Client connection negotiation failed: client connection to 127.8.241.253:37133: connect: Connection refused (error 111)
W20260806 18:44:24.049671 10542 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:44:24.049947 10542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:24.049969 10542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:24.050009 10542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:24.054101 10542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:24.054159 10542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:24.054178 10542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:24.054195 10542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:24.054209 10542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:24.059584 10542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:46271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:36407,127.8.241.253:37133,127.8.241.252:34377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab
--principal=kudu/127.8.241.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.253:37133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.241.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:24.060621 10542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:24.061681 10542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:24.070122 10548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:24.070263 10552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:24.070288 10542 server_base.cc:1034] running on GCE node
W20260806 18:44:24.070122 10549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:24.070960 10542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:24.072033 10542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:24.073194 10542 hybrid_clock.cc:648] HybridClock initialized: now 1786041864073162 us; error 44 us; skew 500 ppm
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:24.075937 10542 init.cc:377] Logged in from keytab as kudu/127.8.241.253@KRBTEST.COM (short username kudu)
I20260806 18:44:24.076920 10542 webserver.cc:460] Webserver started at http://127.8.241.253:45613/ using document root <none> and password file <none>
I20260806 18:44:24.077543 10542 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:24.077615 10542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:24.077838 10542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:24.079994 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/data/instance:
uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33"
format_stamp: "Formatted at 2026-08-06 18:44:24 on dist-test-slave-k5rr"
server_key: "4221dbaf3b8b0546fc66659d6954e167"
server_key_iv: "addddd2f53c8b427142619a86c48eaef"
server_key_version: "encryptionkey@0"
I20260806 18:44:24.080551 10542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/wal/instance:
uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33"
format_stamp: "Formatted at 2026-08-06 18:44:24 on dist-test-slave-k5rr"
server_key: "4221dbaf3b8b0546fc66659d6954e167"
server_key_iv: "addddd2f53c8b427142619a86c48eaef"
server_key_version: "encryptionkey@0"
I20260806 18:44:24.084311 10542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:44:24.086726 10562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:24.087929 10542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:44:24.088040 10542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/wal
uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33"
format_stamp: "Formatted at 2026-08-06 18:44:24 on dist-test-slave-k5rr"
server_key: "4221dbaf3b8b0546fc66659d6954e167"
server_key_iv: "addddd2f53c8b427142619a86c48eaef"
server_key_version: "encryptionkey@0"
I20260806 18:44:24.088133 10542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:24.100576 10542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:24.103668 10542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:24.103863 10542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:24.110944 10542 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.253:37133
I20260806 18:44:24.111193 10636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.253:37133 every 8 connection(s)
I20260806 18:44:24.111940 10542 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
I20260806 18:44:24.114877 10639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:24.118921 10639 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:24.120760 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10542
I20260806 18:44:24.120875 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-1/wal/instance
I20260806 18:44:24.121170 9159 external_mini_cluster.cc:1428] Setting key 680bf18511a12f6cd64c4fb7437ecb4d
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.252@KRBTEST.COM" created.
I20260806 18:44:24.133998 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.120811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.8.241.254:36407)
0806 18:44:24.120972 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.120976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.121861 (+ 885us) server_negotiation.cc:407] Connection header received
0806 18:44:24.122794 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.122798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.122864 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.122936 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.124059 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.124595 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.126276 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.126532 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.126625 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.129545 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.129548 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.129550 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.129575 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.131570 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.132145 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.132149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.132150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.132302 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.132698 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.132701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.132702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.132989 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.133122 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.133684 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.133833 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:44:24.134760 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.121201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36407 (local address 127.0.0.1:46490)
0806 18:44:24.121687 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.121739 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:24.122591 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:24.123109 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:24.123116 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:24.123447 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:24.123921 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.123934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.124731 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.124735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.126146 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.126159 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.126698 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.126703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.126926 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.127539 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:24.127561 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:24.129355 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:24.131679 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.131685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.131701 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.131972 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.132399 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.132402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.132404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.132600 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.133118 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:24.133124 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:24.133410 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 18:44:24.133574 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 18:44:24.133867 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260806 18:44:24.136242 10639 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:24.140188 10534 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } attempt: 3
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041863, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
I20260806 18:44:24.146000 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.136815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:37133 (local address 127.0.0.1:48864)
0806 18:44:24.136928 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.136943 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:24.137051 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:24.137605 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:24.137608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:24.137793 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:24.137885 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.137890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.138832 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.138836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.139376 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.139384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.139953 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.139956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.140066 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.140540 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:24.140556 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:24.142000 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:24.144182 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.144185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.144187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.144462 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.144783 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.144785 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.144787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.144869 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.145431 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:24.145434 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:24.145540 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 18:44:24.145657 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 18:44:24.145795 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":2}
I20260806 18:44:24.146384 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.136889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48864 (local address 127.8.241.253:37133)
0806 18:44:24.137252 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.137261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.137280 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:44:24.137313 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.137318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.137489 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.137609 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.137987 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.138726 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.139479 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.139849 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.139941 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.142130 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.142142 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.142159 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.142185 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.144061 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.144564 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.144570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.144571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.144666 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.144954 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.144957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.144958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.145312 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.145435 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.145763 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.145909 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":178,"threads_started":1}
I20260806 18:44:24.148573 10639 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab.
Entry for principal kudu/127.8.241.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab.
I20260806 18:44:24.149349 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.149045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:41980)
0806 18:44:24.149187 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.149275 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:44:24.149551 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.140649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:37133 (local address 127.0.0.1:48876)
0806 18:44:24.140850 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.140864 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:24.140942 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:24.141453 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:24.141457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:24.141630 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:24.141760 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.141770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.142741 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.142744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.143448 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.143462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.144126 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.144130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.144278 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.144699 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:24.144715 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:24.146345 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:24.147964 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.147968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.147970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.148235 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.148548 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.148551 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.148553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.148649 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.149029 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:24.149032 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:24.149140 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 18:44:24.149258 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:44:24.149407 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
W20260806 18:44:24.149583 10563 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
I20260806 18:44:24.150350 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.252:34377
--webserver_interface=127.8.241.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab
--principal=kudu/127.8.241.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:46271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:36407,127.8.241.253:37133,127.8.241.252:34377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:44:24.150393 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.140701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48876 (local address 127.8.241.253:37133)
0806 18:44:24.141192 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.141201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.141223 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:44:24.141259 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.141262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.141306 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.141391 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.141895 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.142619 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.143591 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.143955 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.144093 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.146494 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.146498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.146501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.146526 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.147866 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.148379 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.148382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.148384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.148457 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.148742 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.148746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.148748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.148912 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.148995 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.149347 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.149459 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":90,"threads_started":1}
I20260806 18:44:24.150475 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.150209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:41994)
0806 18:44:24.150325 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.150389 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 18:44:24.150612 10639 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34377: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
I20260806 18:44:24.151021 10534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:24.151664 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.151422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42006)
0806 18:44:24.151544 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.151596 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 18:44:24.152371 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.152094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42008)
0806 18:44:24.152266 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.152310 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260806 18:44:24.152524 10534 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34377: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
I20260806 18:44:24.183689 10534 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } attempt: 1
I20260806 18:44:24.184525 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.184241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42012)
0806 18:44:24.184384 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.184445 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:44:24.185231 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.184993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42024)
0806 18:44:24.185111 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.185172 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 18:44:24.185348 10534 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34377: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
I20260806 18:44:24.199839 10639 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } attempt: 1
I20260806 18:44:24.200626 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.200356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42038)
0806 18:44:24.200492 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.200551 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 18:44:24.201321 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.201122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42048)
0806 18:44:24.201220 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.201264 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 18:44:24.201472 10639 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34377: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
I20260806 18:44:24.260516 10534 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } attempt: 2
I20260806 18:44:24.261466 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.261110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42064)
0806 18:44:24.261297 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.261373 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 18:44:24.262166 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.261952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42072)
0806 18:44:24.262048 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.262097 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 18:44:24.262292 10534 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34377: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
W20260806 18:44:24.265862 10649 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:44:24.266130 10649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:24.266192 10649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:24.266258 10649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:44:24.269606 10639 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } attempt: 2
I20260806 18:44:24.270290 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.270041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42076)
0806 18:44:24.270169 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.270222 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 18:44:24.270326 10649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:24.270371 10649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:24.270416 10649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:24.270444 10649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:24.270470 10649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:24.271011 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.270805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42090)
0806 18:44:24.270902 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.270944 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 18:44:24.271134 10639 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34377: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34377: connect: Connection refused (error 111)
I20260806 18:44:24.276132 10649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:46271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:36407,127.8.241.253:37133,127.8.241.252:34377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab
--principal=kudu/127.8.241.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.252:34377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.241.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:24.277338 10649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:24.278406 10649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:24.286798 10661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:24.286989 10649 server_base.cc:1034] running on GCE node
W20260806 18:44:24.286804 10655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:24.286916 10657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:24.287648 10649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:24.288713 10649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:24.289883 10649 hybrid_clock.cc:648] HybridClock initialized: now 1786041864289854 us; error 41 us; skew 500 ppm
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:24.292562 10649 init.cc:377] Logged in from keytab as kudu/127.8.241.252@KRBTEST.COM (short username kudu)
I20260806 18:44:24.293454 10649 webserver.cc:460] Webserver started at http://127.8.241.252:34151/ using document root <none> and password file <none>
I20260806 18:44:24.293998 10649 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:24.294064 10649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:24.294272 10649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:24.296279 10649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/data/instance:
uuid: "b9b9d2f8064d4419a99375346c3c6059"
format_stamp: "Formatted at 2026-08-06 18:44:24 on dist-test-slave-k5rr"
server_key: "de10108ddfe91167addf640dd1382e25"
server_key_iv: "9fdfe16adb6cbe1ad7ba15d0472fe2aa"
server_key_version: "encryptionkey@0"
I20260806 18:44:24.296833 10649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/wal/instance:
uuid: "b9b9d2f8064d4419a99375346c3c6059"
format_stamp: "Formatted at 2026-08-06 18:44:24 on dist-test-slave-k5rr"
server_key: "de10108ddfe91167addf640dd1382e25"
server_key_iv: "9fdfe16adb6cbe1ad7ba15d0472fe2aa"
server_key_version: "encryptionkey@0"
I20260806 18:44:24.300400 10649 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:44:24.302739 10675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:24.303932 10649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:44:24.304044 10649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/wal
uuid: "b9b9d2f8064d4419a99375346c3c6059"
format_stamp: "Formatted at 2026-08-06 18:44:24 on dist-test-slave-k5rr"
server_key: "de10108ddfe91167addf640dd1382e25"
server_key_iv: "9fdfe16adb6cbe1ad7ba15d0472fe2aa"
server_key_version: "encryptionkey@0"
I20260806 18:44:24.304132 10649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:24.319482 10649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:24.322335 10649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:24.322510 10649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:24.329485 10649 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.252:34377
I20260806 18:44:24.329746 10746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.252:34377 every 8 connection(s)
I20260806 18:44:24.330449 10649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
I20260806 18:44:24.333386 10747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:24.337487 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10649
I20260806 18:44:24.337600 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041577757111-9159-0/minicluster-data/master-2/wal/instance
I20260806 18:44:24.337332 10747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:24.337869 9159 external_mini_cluster.cc:1428] Setting key f43a3aa7f5c33b4d87f54e27fb12040f
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
I20260806 18:44:24.350495 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.339279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46502 (local address 127.8.241.254:36407)
0806 18:44:24.339417 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.339423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.339816 (+ 393us) server_negotiation.cc:407] Connection header received
0806 18:44:24.340606 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.340609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.340656 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.340746 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.341858 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.342626 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.343925 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.344223 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.344321 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.346826 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.346829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.346831 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.346856 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.348334 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.348887 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.348891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.348901 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.348988 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.349352 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.349355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.349357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.349582 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.349672 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.350194 (+ 522us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.350327 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:44:24.351365 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.339218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:36407 (local address 127.0.0.1:46502)
0806 18:44:24.339672 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.339721 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:24.340414 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:24.340804 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:24.340810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:24.341340 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:24.341720 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.341737 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.342728 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.342731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.343806 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.343818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.344371 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.344374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.344520 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.345143 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:24.345159 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:24.346683 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:24.348437 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.348445 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.348453 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.348797 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.349085 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.349088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.349090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.349269 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.349680 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:24.349686 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:24.349947 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 18:44:24.350117 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 18:44:24.350442 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":120,"threads_started":1}
I20260806 18:44:24.351367 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.339219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42094 (local address 127.8.241.252:34377)
0806 18:44:24.339655 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.339670 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.339703 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:44:24.340404 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.340422 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.340635 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.340802 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.341727 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.342879 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.344033 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.344524 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.344600 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.347188 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.347193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.347205 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.347233 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.348838 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.349300 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.349304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.349306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.349403 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.349661 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.349664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.349665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.349942 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.350100 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.350388 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.350541 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"spinlock_wait_cycles":17792,"thread_start_us":158,"threads_started":1}
I20260806 18:44:24.352620 10747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:44:24.361444 10534 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } attempt: 3
I20260806 18:44:24.362066 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.353142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:37133 (local address 127.0.0.1:48880)
0806 18:44:24.353298 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.353311 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:24.353375 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:24.353619 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:24.353622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:24.353779 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:24.353905 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.353913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.354896 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.354899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.355684 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.355693 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.356352 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.356355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.356421 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.356823 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:24.356837 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:24.358331 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:24.360046 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.360048 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.360050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.360315 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.360615 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.360617 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.360618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.360706 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.361512 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:24.361515 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:24.361623 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 18:44:24.361745 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 18:44:24.361890 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 18:44:24.362424 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.353231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48880 (local address 127.8.241.253:37133)
0806 18:44:24.353360 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.353364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.353378 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:44:24.353457 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.353460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.353509 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.353584 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.354016 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.354794 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.355810 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.356232 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.356366 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.358473 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.358477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.358480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.358512 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.359921 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.360415 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.360419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.360421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.360516 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.360798 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.360802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.360804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.361065 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.361156 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.361850 (+ 694us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.362295 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:44:24.362403 10639 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } attempt: 3
I20260806 18:44:24.363133 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.353790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46514 (local address 127.8.241.254:36407)
0806 18:44:24.353888 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.353892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.353920 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:44:24.354062 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.354064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.354102 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.354157 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.354791 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.355505 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.356558 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.356802 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.356900 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.359473 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.359477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.359480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.359506 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.361056 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.361461 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.361463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.361464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.361534 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.361829 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.361832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.361834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.362038 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.362267 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.362866 (+ 599us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.362984 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:44:24.363489 10747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041863, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041864, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
Aug 06 18:44:24 dist-test-slave-k5rr krb5kdc[10403](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
I20260806 18:44:24.372165 10538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.361876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42100)
0806 18:44:24.362000 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.362016 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:24.362098 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:24.363024 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:24.363028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:24.363174 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:24.363271 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.363277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.364561 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.364571 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.365313 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.365327 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.366099 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.366104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.366194 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.366670 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:24.366686 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:24.368196 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:24.370548 (+ 2352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.370552 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.370554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.370789 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.371109 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.371112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.371114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.371201 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.371695 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:24.371699 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:24.371795 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 18:44:24.371888 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:44:24.372007 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 18:44:24.372686 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.362120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42100 (local address 127.8.241.252:34377)
0806 18:44:24.362803 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.362807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.362818 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:44:24.362860 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.362862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.362902 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.363374 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.363463 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.364130 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.365476 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.365869 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.365977 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.368340 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.368344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.368347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.368373 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.370432 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.370913 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.370916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.370918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.371003 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.371299 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.371303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.371305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.371553 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.371648 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.372405 (+ 757us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.372520 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"spinlock_wait_cycles":4480}
I20260806 18:44:24.374848 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.364465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.8.241.253:37133)
0806 18:44:24.365127 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.365131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.365142 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:44:24.365172 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.365173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.365204 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.365262 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.365978 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.366715 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.367804 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.368118 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.368208 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.371725 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.371729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.371732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.371762 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.373259 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.373708 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.373711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.373713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.373799 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.374085 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.374089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.374091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.374262 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.374478 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.374593 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.374704 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593}
I20260806 18:44:24.374920 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.363969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42116)
0806 18:44:24.364118 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.364129 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:24.364191 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:24.365668 (+ 1477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:24.365671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:24.365808 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:24.365914 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.365921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.366958 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.366961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.367787 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.367798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.368379 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.368382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.368446 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.369047 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:24.369065 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:24.370605 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:24.372962 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.372965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.372967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.373544 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.373873 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.373876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.373877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.373968 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.374459 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:24.374462 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:24.374554 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:44:24.374637 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:44:24.374759 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":23}
I20260806 18:44:24.375623 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.364391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42116 (local address 127.8.241.252:34377)
0806 18:44:24.365456 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.365465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.365476 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:44:24.365509 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.365511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.365549 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.366004 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.366112 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.366863 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.367953 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.368266 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.368592 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.370740 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.370744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.370747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.370769 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.372840 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.373667 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.373670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.373672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.373764 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.374074 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.374076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.374078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.374343 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.375223 (+ 880us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.375307 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.375422 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1002,"thread_start_us":105,"threads_started":1}
I20260806 18:44:24.376641 10534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4: Bootstrap starting.
I20260806 18:44:24.378610 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:24.379251 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.362824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34377 (local address 127.0.0.1:42114)
0806 18:44:24.363127 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:24.363142 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:24.363218 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:24.364784 (+ 1566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:24.364787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:24.364916 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:24.364993 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.364998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.366464 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.366468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.367258 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:24.367272 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.368239 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.368244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:24.368335 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.368771 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:24.368782 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:24.371100 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:24.377681 (+ 6581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.377685 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.377687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.377911 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.378207 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:24.378211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:24.378213 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:24.378301 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:24.378791 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:24.378794 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:24.378893 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 18:44:24.378973 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 18:44:24.379104 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260806 18:44:24.379546 10747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059: Bootstrap starting.
I20260806 18:44:24.379760 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.363496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42114 (local address 127.8.241.252:34377)
0806 18:44:24.364564 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.364579 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.364593 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:44:24.364628 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.364631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.364673 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.364770 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.365107 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.366324 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.367425 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.368080 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.368554 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.371293 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.371297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.371300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.371342 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.377556 (+ 6214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.378023 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.378026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.378027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.378108 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.378474 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.378477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.378478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.378677 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.379421 (+ 744us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.379495 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.379604 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":860,"spinlock_wait_cycles":3840,"thread_start_us":121,"threads_started":1}
I20260806 18:44:24.379849 10534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:24.380815 10534 log.cc:824] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:24.380786 10495 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "69dd35d4cd53422c8b44e7b1f35489f4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46514
I20260806 18:44:24.382794 10639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33: Bootstrap starting.
I20260806 18:44:24.383224 10534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4: No bootstrap required, opened a new log
I20260806 18:44:24.384907 10639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:24.385766 10639 log.cc:824] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:24.386385 10747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:24.386437 10534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } }
I20260806 18:44:24.386806 10534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69dd35d4cd53422c8b44e7b1f35489f4, State: Initialized, Role: FOLLOWER
I20260806 18:44:24.387181 10534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } }
I20260806 18:44:24.387250 10747 log.cc:824] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:24.388000 10639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33: No bootstrap required, opened a new log
I20260806 18:44:24.388643 10534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:24.388734 10495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:44:24.388860 10495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:24.389635 10747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059: No bootstrap required, opened a new log
I20260806 18:44:24.389905 10495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } }
I20260806 18:44:24.390793 10495 leader_election.cc:288] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [CANDIDATE]: Term 1 election: Requested vote from peers 73b0bc3aab5f4b2ea4bd7901916d8d33 (127.8.241.253:37133), b9b9d2f8064d4419a99375346c3c6059 (127.8.241.252:34377)
I20260806 18:44:24.391857 10639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } }
I20260806 18:44:24.392061 10639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b0bc3aab5f4b2ea4bd7901916d8d33, State: Initialized, Role: FOLLOWER
W20260806 18:44:24.392333 10454 tablet.cc:2369] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:44:24.392947 10639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } }
I20260806 18:44:24.394424 10747 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } }
I20260806 18:44:24.394649 10747 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9b9d2f8064d4419a99375346c3c6059, State: Initialized, Role: FOLLOWER
I20260806 18:44:24.394955 10747 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } }
I20260806 18:44:24.396262 10747 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:24.396366 10713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b9b9d2f8064d4419a99375346c3c6059"
I20260806 18:44:24.396606 10713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:24.398213 10713 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69dd35d4cd53422c8b44e7b1f35489f4 in term 1.
I20260806 18:44:24.398718 10444 leader_election.cc:302] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69dd35d4cd53422c8b44e7b1f35489f4, b9b9d2f8064d4419a99375346c3c6059; no voters:
I20260806 18:44:24.396067 10422 master_runner.cc:186] Error getting master registration for 127.8.241.253:37133: OK, instance_id { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" instance_seqno: 1786041864108919 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:44:24.399354 10422 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:44:24.401883 10639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:24.402624 10603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33"
W20260806 18:44:24.402845 10781 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:24.402913 10781 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:24.402886 10603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:24.403126 10757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.403258 10757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:24.403357 10757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:24.403429 10757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [term 1 LEADER]: Becoming Leader. State: Replica: 69dd35d4cd53422c8b44e7b1f35489f4, State: Running, Role: LEADER
I20260806 18:44:24.403756 10757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } }
I20260806 18:44:24.404371 10603 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69dd35d4cd53422c8b44e7b1f35489f4 in term 1.
I20260806 18:44:24.405851 10757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69dd35d4cd53422c8b44e7b1f35489f4. Latest consensus state: current_term: 1 leader_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.405997 10757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:24.403965 10649 master_runner.cc:186] Error getting master registration for 127.8.241.253:37133: OK, instance_id { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" instance_seqno: 1786041864108919 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:44:24.407178 10649 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:44:24.407604 10826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:24.413061 10782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.413198 10782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:24.413242 10783 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.413385 10783 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:24.415283 10826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 18:44:24.416121 10800 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:24.416191 10800 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:24.417863 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.394615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.8.241.253:37133)
0806 18:44:24.394773 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.394778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.399817 (+ 5039us) server_negotiation.cc:407] Connection header received
0806 18:44:24.400024 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.400027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.400079 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.400132 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.401362 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.402091 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.407308 (+ 5217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.407627 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.408488 (+ 861us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.414119 (+ 5631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.414123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.414126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.414156 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.415759 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.416249 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.416253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.416255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.416348 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.416643 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.416647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.416649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.416873 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.416992 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.417601 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.417716 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 18:44:24.422132 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.395853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46516 (local address 127.8.241.254:36407)
0806 18:44:24.395964 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.395969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.401560 (+ 5591us) server_negotiation.cc:407] Connection header received
0806 18:44:24.401729 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.401732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.401780 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.401828 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.409487 (+ 7659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.410165 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.411211 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.411534 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.411606 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.415663 (+ 4057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.415668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.415671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.415701 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.417527 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.420563 (+ 3036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.420567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.420569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.420669 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.421025 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.421030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.421032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.421281 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.421402 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.421809 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.421958 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:44:24.429590 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:24.400344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.8.241.252:34377)
0806 18:44:24.400546 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.400551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.413836 (+ 13285us) server_negotiation.cc:407] Connection header received
0806 18:44:24.413966 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.413969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.414019 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.414095 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.420897 (+ 6802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.421744 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.422877 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:24.423504 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:24.423609 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:24.425482 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:24.425487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:24.425490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:24.425521 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:24.427227 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.427737 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.427741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.427743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.427836 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:24.428140 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:24.428144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:24.428146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:24.428383 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:24.428520 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:24.429281 (+ 761us) server_negotiation.cc:299] Negotiation successful
0806 18:44:24.429412 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
W20260806 18:44:24.429620 10823 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:24.429708 10823 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:24.436888 10713 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [term 1 FOLLOWER]: Refusing update from remote peer 69dd35d4cd53422c8b44e7b1f35489f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:24.437884 10801 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:24.444679 10603 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [term 1 FOLLOWER]: Refusing update from remote peer 69dd35d4cd53422c8b44e7b1f35489f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:24.445295 10801 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:24.449393 10782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69dd35d4cd53422c8b44e7b1f35489f4. Latest consensus state: current_term: 1 leader_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.449537 10782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:24.450372 10783 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69dd35d4cd53422c8b44e7b1f35489f4. Latest consensus state: current_term: 1 leader_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.450532 10783 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:24.457485 10801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.457629 10801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:24.458122 10830 mvcc.cc:205] Tried to move back new op lower bound from 7315627476714672128 to 7315627476601810944. Current Snapshot: MvccSnapshot[applied={T|T < 7315627476714672128}]
I20260806 18:44:24.461580 10782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.461696 10782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:24.463165 10833 mvcc.cc:205] Tried to move back new op lower bound from 7315627476714672128 to 7315627476601810944. Current Snapshot: MvccSnapshot[applied={T|T < 7315627476714672128}]
I20260806 18:44:24.463896 10757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.464022 10757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:24.464761 10826 catalog_manager.cc:1347] Generated new cluster ID: 8c30b540aad6476992fd2255f516dbcc
I20260806 18:44:24.464833 10826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:24.465299 10783 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dd35d4cd53422c8b44e7b1f35489f4" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 36407 } } peers { permanent_uuid: "73b0bc3aab5f4b2ea4bd7901916d8d33" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 37133 } } peers { permanent_uuid: "b9b9d2f8064d4419a99375346c3c6059" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34377 } } }
I20260806 18:44:24.465422 10783 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:24.484568 10826 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:24.486017 10826 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:24.498934 10826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69dd35d4cd53422c8b44e7b1f35489f4: Generated new TSK 0
I20260806 18:44:24.499883 10826 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:44:24.534281 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.530822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48902 (local address 127.8.241.253:37133)
0806 18:44:24.530958 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.530963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.533092 (+ 2129us) server_negotiation.cc:407] Connection header received
0806 18:44:24.533330 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.533334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.533387 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.533440 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.534116 (+ 676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48902: BlockingRecv error: recv got EOF from 127.0.0.1:48902 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 18:44:24.535694 10539 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.531709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.8.241.254:36407)
0806 18:44:24.531829 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.531833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.534778 (+ 2945us) server_negotiation.cc:407] Connection header received
0806 18:44:24.534952 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.534956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.535006 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.535055 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.535570 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46518: BlockingRecv error: recv got EOF from 127.0.0.1:46518 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:44:24.537279 10752 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.532433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42132 (local address 127.8.241.252:34377)
0806 18:44:24.532551 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.532556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.536211 (+ 3655us) server_negotiation.cc:407] Connection header received
0806 18:44:24.536457 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.536461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.536513 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.536563 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.537137 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42132: BlockingRecv error: recv got EOF from 127.0.0.1:42132 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:44:24.541890 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.540304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48908 (local address 127.8.241.253:37133)
0806 18:44:24.540411 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.540415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.540888 (+ 473us) server_negotiation.cc:407] Connection header received
0806 18:44:24.541145 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.541149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.541227 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.541290 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.541787 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48908: BlockingRecv error: recv got EOF from 127.0.0.1:48908 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:44:24.546739 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.545260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48912 (local address 127.8.241.253:37133)
0806 18:44:24.545357 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.545361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.545860 (+ 499us) server_negotiation.cc:407] Connection header received
0806 18:44:24.546036 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.546039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.546087 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.546137 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.546635 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48912: BlockingRecv error: recv got EOF from 127.0.0.1:48912 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 18:44:24.552021 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.550617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48924 (local address 127.8.241.253:37133)
0806 18:44:24.550707 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.550711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.551176 (+ 465us) server_negotiation.cc:407] Connection header received
0806 18:44:24.551356 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.551359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.551404 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.551466 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.551924 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48924: BlockingRecv error: recv got EOF from 127.0.0.1:48924 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 18:44:24.559679 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.558066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.8.241.253:37133)
0806 18:44:24.558195 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.558199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.558739 (+ 540us) server_negotiation.cc:407] Connection header received
0806 18:44:24.558936 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.558940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.558991 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.559038 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.559574 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48932: BlockingRecv error: recv got EOF from 127.0.0.1:48932 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 18:44:24.571801 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.569997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.8.241.253:37133)
0806 18:44:24.570119 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.570123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.570730 (+ 607us) server_negotiation.cc:407] Connection header received
0806 18:44:24.570942 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.570946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.571000 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.571056 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.571671 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48942: BlockingRecv error: recv got EOF from 127.0.0.1:48942 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 18:44:24.592881 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.590566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.8.241.253:37133)
0806 18:44:24.590701 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.590705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.591550 (+ 845us) server_negotiation.cc:407] Connection header received
0806 18:44:24.591882 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.591886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.591949 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.592005 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.592751 (+ 746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48958: BlockingRecv error: recv got EOF from 127.0.0.1:48958 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260806 18:44:24.628439 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.627338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48962 (local address 127.8.241.253:37133)
0806 18:44:24.627479 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.627484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.627603 (+ 119us) server_negotiation.cc:407] Connection header received
0806 18:44:24.627733 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.627737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.627796 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.627893 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.628310 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48962: BlockingRecv error: recv got EOF from 127.0.0.1:48962 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 18:44:24.695633 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.694546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.8.241.253:37133)
0806 18:44:24.694719 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.694725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.694834 (+ 109us) server_negotiation.cc:407] Connection header received
0806 18:44:24.694966 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.694969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.695020 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.695091 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.695495 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48968: BlockingRecv error: recv got EOF from 127.0.0.1:48968 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260806 18:44:24.826982 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:24.825804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.8.241.253:37133)
0806 18:44:24.825984 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:24.825989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:24.826107 (+ 118us) server_negotiation.cc:407] Connection header received
0806 18:44:24.826236 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:24.826239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:24.826297 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:24.826369 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:24.826852 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48980: BlockingRecv error: recv got EOF from 127.0.0.1:48980 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 18:44:25.086128 10646 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:25.085032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.8.241.253:37133)
0806 18:44:25.085163 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:25.085168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:25.085255 (+ 87us) server_negotiation.cc:407] Connection header received
0806 18:44:25.085409 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:25.085413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:25.085468 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:25.085537 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:25.086001 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48996: BlockingRecv error: recv got EOF from 127.0.0.1:48996 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:44:25.417697 10800 catalog_manager.cc:1259] Loaded cluster ID: 8c30b540aad6476992fd2255f516dbcc
I20260806 18:44:25.417790 10800 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059: loading cluster ID for follower catalog manager: success
I20260806 18:44:25.419544 10800 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059: acquiring CA information for follower catalog manager: success
I20260806 18:44:25.420768 10800 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b9b9d2f8064d4419a99375346c3c6059: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:44:25.431111 10823 catalog_manager.cc:1259] Loaded cluster ID: 8c30b540aad6476992fd2255f516dbcc
I20260806 18:44:25.431190 10823 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33: loading cluster ID for follower catalog manager: success
I20260806 18:44:25.432860 10823 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33: acquiring CA information for follower catalog manager: success
I20260806 18:44:25.434080 10823 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 73b0bc3aab5f4b2ea4bd7901916d8d33: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:44:25.603775 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:25.600060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49000 (local address 127.8.241.253:37133)
0806 18:44:25.600302 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:25.600306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:25.600321 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:44:25.600468 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:25.600471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:25.600614 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:25.600719 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:25.600958 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.601502 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.602357 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.602579 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.602655 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:25.602913 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:25.603428 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:25.603524 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:25.603583 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 18:44:25.603634 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":69,"threads_started":1}
I20260806 18:44:25.610342 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:25.606721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42134 (local address 127.8.241.252:34377)
0806 18:44:25.606947 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:25.606952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:25.606967 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:44:25.607058 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:25.607060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:25.607182 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:25.607280 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:25.607564 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.608192 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.609027 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.609256 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.609332 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:25.609588 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:25.610040 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:25.610129 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:25.610163 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 18:44:25.610214 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":63,"threads_started":1}
I20260806 18:44:25.617435 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:25.613450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.8.241.253:37133)
0806 18:44:25.613603 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:25.613608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:25.614141 (+ 533us) server_negotiation.cc:407] Connection header received
0806 18:44:25.614273 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:25.614276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:25.614334 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:25.614391 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:25.614737 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.615398 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.616309 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.616616 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.616717 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:25.616955 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:25.617064 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:25.617133 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:25.617229 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 18:44:25.617291 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:44:25.617862 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:25.613585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42150 (local address 127.8.241.252:34377)
0806 18:44:25.613693 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:25.613697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:25.613855 (+ 158us) server_negotiation.cc:407] Connection header received
0806 18:44:25.613944 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:25.613947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:25.613999 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:25.614063 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:25.614378 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.615065 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.616424 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.616732 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.616833 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:25.617075 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:25.617153 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:25.617224 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:25.617673 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 18:44:25.617726 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:44:25.618765 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:25.613630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46520 (local address 127.8.241.254:36407)
0806 18:44:25.613849 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:25.613852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:25.613865 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:44:25.613949 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:25.613951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:25.614069 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:25.614181 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:25.614486 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.615095 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.616815 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:25.617164 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:25.617262 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:25.617896 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:25.618278 (+ 382us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:25.618388 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:25.618451 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 18:44:25.618520 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":62,"threads_started":1}
I20260806 18:44:25.621534 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10422
I20260806 18:44:25.636369 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10542
I20260806 18:44:25.651417 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10649
2026-08-06T18:44:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3204 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:25 dist-test-slave-k5rr krb5kdc[10958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:25 dist-test-slave-k5rr krb5kdc[10958](info): set up 2 sockets
Aug 06 18:44:25 dist-test-slave-k5rr krb5kdc[10958](info): commencing operation
krb5kdc: starting...
W20260806 18:44:26.775645 9159 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/test-user.keytab.
Aug 06 18:44:26 dist-test-slave-k5rr krb5kdc[10958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.241.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
Entry for principal kudu/127.8.241.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab.
I20260806 18:44:26.894088 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39023
--webserver_interface=127.8.241.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39023,127.8.241.253:36693,127.8.241.252:34435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:27.006603 10977 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:44:27.006927 10977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:27.006963 10977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:27.007027 10977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:27.011037 10977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:27.011102 10977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:27.011130 10977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:27.011153 10977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:27.011183 10977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:27.016494 10977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39023,127.8.241.253:36693,127.8.241.252:34435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.254.keytab
--principal=kudu/127.8.241.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.254:39023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.241.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:27.017611 10977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:27.018637 10977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:27.026894 10984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:27.026930 10986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:27.026911 10983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:27.027248 10977 server_base.cc:1034] running on GCE node
I20260806 18:44:27.027864 10977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:27.029050 10977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:27.030251 10977 hybrid_clock.cc:648] HybridClock initialized: now 1786041867030213 us; error 40 us; skew 500 ppm
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:27.032938 10977 init.cc:377] Logged in from keytab as kudu/127.8.241.254@KRBTEST.COM (short username kudu)
I20260806 18:44:27.033886 10977 webserver.cc:460] Webserver started at http://127.8.241.254:40513/ using document root <none> and password file <none>
I20260806 18:44:27.034448 10977 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:27.034515 10977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:27.034726 10977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:27.036832 10977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/data/instance:
uuid: "a09e162ce8d8414fadf4d947887f79e9"
format_stamp: "Formatted at 2026-08-06 18:44:27 on dist-test-slave-k5rr"
server_key: "23c28ac58e0a05eddcbaa2f8c1fa8886"
server_key_iv: "395dbf04f578429f20e6331c6b1a8787"
server_key_version: "encryptionkey@0"
I20260806 18:44:27.037424 10977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/wal/instance:
uuid: "a09e162ce8d8414fadf4d947887f79e9"
format_stamp: "Formatted at 2026-08-06 18:44:27 on dist-test-slave-k5rr"
server_key: "23c28ac58e0a05eddcbaa2f8c1fa8886"
server_key_iv: "395dbf04f578429f20e6331c6b1a8787"
server_key_version: "encryptionkey@0"
I20260806 18:44:27.040992 10977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:44:27.043378 10996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:27.044589 10977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:44:27.044705 10977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/wal
uuid: "a09e162ce8d8414fadf4d947887f79e9"
format_stamp: "Formatted at 2026-08-06 18:44:27 on dist-test-slave-k5rr"
server_key: "23c28ac58e0a05eddcbaa2f8c1fa8886"
server_key_iv: "395dbf04f578429f20e6331c6b1a8787"
server_key_version: "encryptionkey@0"
I20260806 18:44:27.044801 10977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:27.062913 10977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:27.065656 10977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:27.065841 10977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:27.072623 10977 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.254:39023
I20260806 18:44:27.072887 11080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.254:39023 every 8 connection(s)
I20260806 18:44:27.073616 10977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:27.076462 11081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:27.079906 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 10977
I20260806 18:44:27.080037 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-0/wal/instance
I20260806 18:44:27.080344 9159 external_mini_cluster.cc:1428] Setting key 09e8a0efa4202fc7f69088d2ebd0a2ac
I20260806 18:44:27.080485 11081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.241.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.253@KRBTEST.COM" created.
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:44:27.098088 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.083307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39023 (local address 127.0.0.1:35378)
0806 18:44:27.083822 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.083872 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:27.084734 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:27.085520 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:27.085526 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:27.086004 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:27.086370 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.086391 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.087518 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.087522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.088672 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.088685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.089377 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.089381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.089497 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.090029 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:27.090046 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:27.092985 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:27.095162 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.095171 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.095180 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.095407 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.095747 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.095750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.095751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.095848 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.096604 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:27.096610 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:27.096737 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 18:44:27.096889 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 18:44:27.097206 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":146,"threads_started":1}
I20260806 18:44:27.098090 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.084257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.8.241.254:39023)
0806 18:44:27.084565 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.084575 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.084620 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:44:27.085065 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.085070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.085263 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.085458 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.086524 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.087385 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.088814 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.089228 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.089342 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.093150 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.093156 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.093167 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.093190 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.095048 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.095517 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.095524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.095526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.095664 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.095927 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.095930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.095932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.096483 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.096624 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.097027 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.097326 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":201,"thread_start_us":94,"threads_started":1}
I20260806 18:44:27.100733 11081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:27.101439 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.101194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:36693 (local address 127.0.0.1:41934)
0806 18:44:27.101307 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.101377 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:44:27.101627 10997 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
I20260806 18:44:27.102380 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.102174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:36693 (local address 127.0.0.1:41948)
0806 18:44:27.102283 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.102327 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:44:27.102519 11081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.253:36693: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab.
Entry for principal kudu/127.8.241.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab.
I20260806 18:44:27.108305 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.253:36693
--webserver_interface=127.8.241.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab
--principal=kudu/127.8.241.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39023,127.8.241.253:36693,127.8.241.252:34435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:44:27.151741 11081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } attempt: 1
I20260806 18:44:27.152572 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.152273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:36693 (local address 127.0.0.1:41964)
0806 18:44:27.152424 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.152488 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 18:44:27.153370 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.153156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:36693 (local address 127.0.0.1:41966)
0806 18:44:27.153257 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.153306 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:44:27.153494 11081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.253:36693: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
I20260806 18:44:27.221642 11081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } attempt: 2
I20260806 18:44:27.222496 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.222206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:36693 (local address 127.0.0.1:41976)
0806 18:44:27.222350 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.222416 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 18:44:27.223179 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.222941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:36693 (local address 127.0.0.1:41988)
0806 18:44:27.223062 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.223114 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 18:44:27.223106 11088 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:44:27.223436 11088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:27.223426 11081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.253:36693: Network error: Client connection negotiation failed: client connection to 127.8.241.253:36693: connect: Connection refused (error 111)
W20260806 18:44:27.223487 11088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:27.223551 11088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:27.227833 11088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:27.227906 11088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:27.227934 11088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:27.227957 11088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:27.227986 11088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:27.233385 11088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39023,127.8.241.253:36693,127.8.241.252:34435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.253.keytab
--principal=kudu/127.8.241.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.253:36693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.241.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:27.234432 11088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:27.235471 11088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:27.243837 11093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:27.243893 11088 server_base.cc:1034] running on GCE node
W20260806 18:44:27.243839 11094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:27.243832 11096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:27.244630 11088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:27.245803 11088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:27.246991 11088 hybrid_clock.cc:648] HybridClock initialized: now 1786041867246966 us; error 33 us; skew 500 ppm
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:27.250170 11088 init.cc:377] Logged in from keytab as kudu/127.8.241.253@KRBTEST.COM (short username kudu)
I20260806 18:44:27.251420 11088 webserver.cc:460] Webserver started at http://127.8.241.253:44059/ using document root <none> and password file <none>
I20260806 18:44:27.252208 11088 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:27.252306 11088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:27.252575 11088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:27.255577 11088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/data/instance:
uuid: "f55b38f9af584c07b5fa62521ab552ea"
format_stamp: "Formatted at 2026-08-06 18:44:27 on dist-test-slave-k5rr"
server_key: "833ad47749c46a7169196e4376697fea"
server_key_iv: "bac5d350f8972531da577469f44b01f5"
server_key_version: "encryptionkey@0"
I20260806 18:44:27.256532 11088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/wal/instance:
uuid: "f55b38f9af584c07b5fa62521ab552ea"
format_stamp: "Formatted at 2026-08-06 18:44:27 on dist-test-slave-k5rr"
server_key: "833ad47749c46a7169196e4376697fea"
server_key_iv: "bac5d350f8972531da577469f44b01f5"
server_key_version: "encryptionkey@0"
I20260806 18:44:27.261247 11088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:44:27.263700 11108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:27.265094 11088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:27.265237 11088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/wal
uuid: "f55b38f9af584c07b5fa62521ab552ea"
format_stamp: "Formatted at 2026-08-06 18:44:27 on dist-test-slave-k5rr"
server_key: "833ad47749c46a7169196e4376697fea"
server_key_iv: "bac5d350f8972531da577469f44b01f5"
server_key_version: "encryptionkey@0"
I20260806 18:44:27.265343 11088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:27.275564 11088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:27.278337 11088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:27.278513 11088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:27.285305 11088 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.253:36693
I20260806 18:44:27.285565 11189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.253:36693 every 8 connection(s)
I20260806 18:44:27.286259 11088 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/data/info.pb
I20260806 18:44:27.288993 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:27.292891 11190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:27.294759 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 11088
I20260806 18:44:27.294876 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-1/wal/instance
I20260806 18:44:27.295151 9159 external_mini_cluster.cc:1428] Setting key a910fe5d63ee405b433344695c4355c0
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.241.252@KRBTEST.COM; defaulting to no policy
I20260806 18:44:27.306926 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.294803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35388 (local address 127.8.241.254:39023)
0806 18:44:27.294952 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.294957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.295881 (+ 924us) server_negotiation.cc:407] Connection header received
0806 18:44:27.296820 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.296824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.296887 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.296959 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.298194 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.298719 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.300385 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.300618 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.300699 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.303445 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.303448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.303450 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.303475 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.304771 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.305450 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.305453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.305454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.305528 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.305839 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.305841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.305842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.306017 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.306089 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.306680 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.306784 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.241.252@KRBTEST.COM" created.
I20260806 18:44:27.307823 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.295188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39023 (local address 127.0.0.1:35388)
0806 18:44:27.295705 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.295759 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:27.296616 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:27.297178 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:27.297185 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:27.297553 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:27.298037 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.298055 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.298855 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.298859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.300243 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.300256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.300777 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.300782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.300997 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.301598 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:27.301619 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:27.303267 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:27.304876 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.304881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.304893 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.305356 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.305607 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.305609 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.305611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.305760 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.306136 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:27.306151 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:27.306447 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 18:44:27.306616 (+ 169us) client_negotiation.cc:240] Negotiation successful
0806 18:44:27.306957 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":123,"threads_started":1}
I20260806 18:44:27.309293 11190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:27.314581 11081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } attempt: 3
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
I20260806 18:44:27.318848 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.309795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:36693 (local address 127.0.0.1:42004)
0806 18:44:27.309922 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.309936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:27.310013 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:27.310628 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:27.310630 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:27.310761 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:27.310855 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.310859 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.312195 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.312197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.312735 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.312743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.313555 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.313558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.313638 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.314043 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:27.314054 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:27.315473 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:27.317256 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.317259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.317261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.317449 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.317732 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.317734 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.317736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.317808 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.318401 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:27.318404 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:27.318496 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:44:27.318606 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:44:27.318710 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 18:44:27.319357 11199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.309894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.8.241.253:36693)
0806 18:44:27.310232 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.310242 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.310264 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:44:27.310304 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.310309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.310498 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.310625 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.310954 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.312078 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.312847 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.313417 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.313520 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.315602 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.315613 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.315629 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.315654 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.317168 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.317547 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.317554 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.317555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.317642 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.317902 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.317906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.317908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.318284 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.318426 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.318702 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.318909 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":245,"thread_start_us":157,"threads_started":1}
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
I20260806 18:44:27.321121 11190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:27.321851 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.321599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59058)
0806 18:44:27.321699 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.321783 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:44:27.322045 11120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
I20260806 18:44:27.322901 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.322705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59060)
0806 18:44:27.322808 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.322850 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:44:27.323072 11190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34435: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.241.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab.
Entry for principal kudu/127.8.241.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab.
I20260806 18:44:27.323868 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.315070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.8.241.253:36693)
0806 18:44:27.315385 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.315394 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.315405 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:44:27.315432 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.315433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.315474 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.315545 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.316131 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.316821 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.317679 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.318003 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.318114 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.320483 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.320486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.320488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.320508 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.322299 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.322762 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.322766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.322768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.322850 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.323117 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.323120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.323122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.323334 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.323423 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.323664 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.323754 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":254,"thread_start_us":97,"threads_started":1}
I20260806 18:44:27.323890 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.315002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:36693 (local address 127.0.0.1:42010)
0806 18:44:27.315177 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.315191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:27.315289 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:27.315578 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:27.315581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:27.315729 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:27.315825 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.315831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.316925 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.316928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.317573 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.317583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.318110 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.318114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.318222 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.318633 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:27.318647 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:27.320340 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:27.322409 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.322413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.322415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.322659 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.322935 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.322938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.322940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.323035 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.323420 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:27.323422 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:27.323510 (+ 88us) client_negotiation.cc:769] Sending connection context
0806 18:44:27.323607 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 18:44:27.323739 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 18:44:27.324213 9159 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5RraP2/build/debug/bin/kudu
/tmp/dist-test-task5RraP2/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.252:34435
--webserver_interface=127.8.241.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab
--principal=kudu/127.8.241.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.241.212:33319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.241.254:39023,127.8.241.253:36693,127.8.241.252:34435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:44:27.324522 11081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:27.325209 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.324927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59070)
0806 18:44:27.325087 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.325140 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 18:44:27.325860 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.325634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59072)
0806 18:44:27.325762 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.325807 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260806 18:44:27.325982 11081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34435: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
I20260806 18:44:27.357146 11081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } attempt: 1
I20260806 18:44:27.358058 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.357714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59082)
0806 18:44:27.357881 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.357956 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 18:44:27.358804 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.358568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59098)
0806 18:44:27.358685 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.358739 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:44:27.358942 11081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34435: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
I20260806 18:44:27.372303 11190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } attempt: 1
I20260806 18:44:27.373186 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.372816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59104)
0806 18:44:27.372991 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.373091 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 18:44:27.373880 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.373666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59106)
0806 18:44:27.373762 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.373815 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 18:44:27.374015 11190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34435: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
I20260806 18:44:27.434109 11081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } attempt: 2
I20260806 18:44:27.434986 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.434643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59118)
0806 18:44:27.434810 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.434886 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 18:44:27.435693 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.435471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59134)
0806 18:44:27.435571 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.435624 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 18:44:27.435832 11081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34435: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
W20260806 18:44:27.441254 11202 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:44:27.441510 11202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:27.441540 11202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:27.441581 11202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:44:27.442166 11190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } attempt: 2
I20260806 18:44:27.442873 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.442602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59146)
0806 18:44:27.442737 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.442796 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 18:44:27.443536 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.443328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59158)
0806 18:44:27.443424 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.443472 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 18:44:27.443648 11190 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.241.252:34435: Network error: Client connection negotiation failed: client connection to 127.8.241.252:34435: connect: Connection refused (error 111)
W20260806 18:44:27.445986 11202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:27.446038 11202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:27.446055 11202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:27.446076 11202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:27.446090 11202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:27.451375 11202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.241.212:33319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.241.254:39023,127.8.241.253:36693,127.8.241.252:34435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/krb5kdc/kudu_127.8.241.252.keytab
--principal=kudu/127.8.241.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.241.252:34435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.241.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:57:37 UTC on ccycloud.quasar-crrbkw.root.comops.site
build id 4571032
I20260806 18:44:27.452488 11202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:27.453543 11202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:27.461871 11209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:27.461849 11211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:27.461860 11207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:27.461951 11202 server_base.cc:1034] running on GCE node
I20260806 18:44:27.462596 11202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:27.463655 11202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:27.464808 11202 hybrid_clock.cc:648] HybridClock initialized: now 1786041867464790 us; error 31 us; skew 500 ppm
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:27.471398 11202 init.cc:377] Logged in from keytab as kudu/127.8.241.252@KRBTEST.COM (short username kudu)
I20260806 18:44:27.472323 11202 webserver.cc:460] Webserver started at http://127.8.241.252:37893/ using document root <none> and password file <none>
I20260806 18:44:27.472899 11202 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:27.472968 11202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:27.473207 11202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:27.475286 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/data/instance:
uuid: "dc98b7e7ff9a4ada9436fec685ce847f"
format_stamp: "Formatted at 2026-08-06 18:44:27 on dist-test-slave-k5rr"
server_key: "c219ac7b70515dd7b4260a9594ad4c7b"
server_key_iv: "e4b477ca409b4780db376e58788230cb"
server_key_version: "encryptionkey@0"
I20260806 18:44:27.475841 11202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/wal/instance:
uuid: "dc98b7e7ff9a4ada9436fec685ce847f"
format_stamp: "Formatted at 2026-08-06 18:44:27 on dist-test-slave-k5rr"
server_key: "c219ac7b70515dd7b4260a9594ad4c7b"
server_key_iv: "e4b477ca409b4780db376e58788230cb"
server_key_version: "encryptionkey@0"
I20260806 18:44:27.480474 11202 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:44:27.483647 11220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:27.485342 11202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 18:44:27.485483 11202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/data,/tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/wal
uuid: "dc98b7e7ff9a4ada9436fec685ce847f"
format_stamp: "Formatted at 2026-08-06 18:44:27 on dist-test-slave-k5rr"
server_key: "c219ac7b70515dd7b4260a9594ad4c7b"
server_key_iv: "e4b477ca409b4780db376e58788230cb"
server_key_version: "encryptionkey@0"
I20260806 18:44:27.485597 11202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:27.509243 11202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:27.512166 11202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:27.512342 11202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:27.519344 11202 rpc_server.cc:307] RPC server started. Bound to: 127.8.241.252:34435
I20260806 18:44:27.519606 11305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.241.252:34435 every 8 connection(s)
I20260806 18:44:27.520318 11202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/data/info.pb
I20260806 18:44:27.520906 9159 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5RraP2/build/debug/bin/kudu as pid 11202
I20260806 18:44:27.521046 9159 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5RraP2/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041577757111-9159-0/minicluster-data/master-2/wal/instance
I20260806 18:44:27.521347 9159 external_mini_cluster.cc:1428] Setting key e83386515a7b77fd9e0c20bfbe876651
I20260806 18:44:27.523463 11306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:27.527706 11306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:44:27.534785 11190 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } attempt: 3
I20260806 18:44:27.534969 11081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } attempt: 3
I20260806 18:44:27.537091 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.522808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.8.241.252:34435)
0806 18:44:27.523215 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.523232 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.523268 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:44:27.523959 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.523978 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.524330 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.524605 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.525581 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.526608 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.527759 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.528288 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.528414 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.531299 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.531305 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.531314 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.531341 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.532940 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.533372 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.533378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.533385 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.533501 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.533805 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.533808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.533809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.534196 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.534373 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.534684 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.535956 (+ 1272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":114,"threads_started":1}
I20260806 18:44:27.538763 11312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.529622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.254:39023 (local address 127.0.0.1:35390)
0806 18:44:27.530083 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.530134 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:27.530268 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:27.530577 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:27.530583 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:27.530975 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:27.531123 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.531130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.531963 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.531966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.532665 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.532675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.533125 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.533129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.533246 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.533665 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:27.533676 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:27.534956 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:27.536832 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.536841 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.536847 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.537235 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.537559 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.537562 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.537564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.537649 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.538119 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:27.538125 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:27.538242 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 18:44:27.538377 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 18:44:27.538553 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":136,"threads_started":1}
I20260806 18:44:27.539412 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.529695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.8.241.254:39023)
0806 18:44:27.529836 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.529841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.530242 (+ 401us) server_negotiation.cc:407] Connection header received
0806 18:44:27.530381 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.530385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.530436 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.530496 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.531246 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.531832 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.532778 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.532991 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.533125 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.535072 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.535076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.535079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.535107 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.536728 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.537358 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.537361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.537363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.537460 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.537775 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.537778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.537780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.537987 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.538068 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.539142 (+ 1074us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.539268 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:44:27.540416 11306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.254@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.253@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
I20260806 18:44:27.544991 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.535827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59180)
0806 18:44:27.536005 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.536018 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:27.536095 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:27.536695 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:27.536698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:27.536864 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:27.536962 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.536968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.538077 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.538081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.538836 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.538847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.539443 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.539447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.539527 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.539944 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:27.539958 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:27.541565 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:27.543473 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.543476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.543478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.543708 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.543989 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.543992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.543994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.544080 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.544528 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:27.544531 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:27.544622 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 18:44:27.544721 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 18:44:27.544853 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 18:44:27.545605 11314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.536230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.8.241.252:34435)
0806 18:44:27.536466 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.536474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.536487 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:44:27.536520 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.536522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.536560 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.537105 (+ 545us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.537237 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.537948 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.538977 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.539310 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.539729 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.541697 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.541700 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.541703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.541725 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.543356 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.543809 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.543813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.543814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.543893 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.544174 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.544178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.544180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.544393 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.544480 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.545338 (+ 858us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.545442 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.254@KRBTEST.COM
I20260806 18:44:27.547407 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.535268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59166)
0806 18:44:27.535468 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.535483 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:27.535567 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:27.536243 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:27.536247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:27.536434 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:27.536551 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.536559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.537968 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.537972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.538769 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.538784 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.540313 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.540316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.540398 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.540872 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:27.540886 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:27.542831 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:27.545447 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.545453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.545455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.545693 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.546371 (+ 678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.546375 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.546377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.546460 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.546909 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:27.546912 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:27.547009 (+ 97us) client_negotiation.cc:769] Sending connection context
0806 18:44:27.547091 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:44:27.547259 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260806 18:44:27.549614 11081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9: Bootstrap starting.
I20260806 18:44:27.549867 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.535415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.8.241.252:34435)
0806 18:44:27.535746 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.535754 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.535768 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:44:27.535802 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.535805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.535863 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.536671 (+ 808us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.536762 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.537842 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.539132 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.540163 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.542548 (+ 2385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.542978 (+ 430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.542981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.542984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.543008 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.544715 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.546151 (+ 1436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.546154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.546156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.546256 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.546550 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.546553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.546555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.546745 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.548376 (+ 1631us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.548481 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.548903 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":262,"thread_start_us":124,"threads_started":1}
I20260806 18:44:27.550944 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.540607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.8.241.254:39023)
0806 18:44:27.541207 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.541211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.541224 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:44:27.541255 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.541257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.541296 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.542272 (+ 976us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.542363 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.543224 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.544338 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.544656 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.545120 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.547336 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.547339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.547341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.547364 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.548787 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.549363 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.549365 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.549366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.549434 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.549688 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.549692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.549693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.549879 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.550602 (+ 723us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.550681 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.550794 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":120,"threads_started":1}
I20260806 18:44:27.552063 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.541245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42018 (local address 127.8.241.253:36693)
0806 18:44:27.541346 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.541351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.541363 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:44:27.541483 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.541487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.541534 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.541583 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.542200 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.542938 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.544003 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.544340 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.544413 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.547089 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.547093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.547096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.547124 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.549657 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.550705 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.550709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.550711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.550799 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.551110 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.551114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.551116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.551381 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.551469 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.551806 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.551923 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 18:44:27.552507 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.540882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.253:36693 (local address 127.0.0.1:42018)
0806 18:44:27.541174 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.541187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:27.541268 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:27.541708 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:27.541711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:27.541959 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:27.542068 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.542075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.543082 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.543085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.543882 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.543891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.544549 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.544552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.544645 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.545083 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:27.545097 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:27.546934 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:27.550306 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.550309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.550311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.550536 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.550912 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.550915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.550917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.551007 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.551541 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:27.551544 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:27.551633 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:44:27.552185 (+ 552us) client_negotiation.cc:240] Negotiation successful
0806 18:44:27.552338 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":127,"threads_started":1}
I20260806 18:44:27.552735 11081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:27.552805 11190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea: Bootstrap starting.
I20260806 18:44:27.553274 11306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:44:27.553704 11081 log.cc:824] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:27.555398 11190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:27.555951 11081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9: No bootstrap required, opened a new log
I20260806 18:44:27.556234 11190 log.cc:824] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041867, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.241.252@KRBTEST.COM for kudu/127.8.241.252@KRBTEST.COM
I20260806 18:44:27.560109 11081 raft_consensus.cc:348] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } }
I20260806 18:44:27.560367 11081 raft_consensus.cc:729] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a09e162ce8d8414fadf4d947887f79e9, State: Initialized, Role: FOLLOWER
I20260806 18:44:27.560597 11190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea: No bootstrap required, opened a new log
I20260806 18:44:27.560714 11081 consensus_queue.cc:260] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } }
I20260806 18:44:27.561393 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.553798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.241.252:34435 (local address 127.0.0.1:59192)
0806 18:44:27.553964 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:27.553977 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:27.554060 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:27.554303 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:27.554306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:27.554468 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:27.554571 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.554577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.555254 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.555257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.555766 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:27.555774 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.556187 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.556191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:27.556345 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.556793 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:27.556804 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:27.558163 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:27.559991 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.559993 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.559995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.560208 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.560465 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:27.560467 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:27.560469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:27.560540 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:27.560887 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:27.560889 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:27.560957 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 18:44:27.561090 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 18:44:27.561217 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":25}
I20260806 18:44:27.561928 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.553859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59192 (local address 127.8.241.252:34435)
0806 18:44:27.553959 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.553963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.554020 (+ 57us) server_negotiation.cc:407] Connection header received
0806 18:44:27.554148 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.554150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.554185 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.554258 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.554675 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.555164 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.555862 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.556097 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.556182 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.558281 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.558285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.558287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.558308 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.559892 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.560308 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.560311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.560313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.560384 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.560611 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.560614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.560616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.560786 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.560859 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.561681 (+ 822us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.561787 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":37}
I20260806 18:44:27.562243 11081 sys_catalog.cc:564] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 18:44:27 dist-test-slave-k5rr krb5kdc[10958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.241.253@KRBTEST.COM
I20260806 18:44:27.565147 11306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f: Bootstrap starting.
I20260806 18:44:27.565533 11190 raft_consensus.cc:348] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } }
W20260806 18:44:27.567359 11004 tablet.cc:2369] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:44:27.567916 11306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:27.568439 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.557893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42030 (local address 127.8.241.253:36693)
0806 18:44:27.557998 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.558002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.558075 (+ 73us) server_negotiation.cc:407] Connection header received
0806 18:44:27.558257 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.558260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.558305 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.558385 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.559305 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.560038 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.561087 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.561392 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.561467 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.564249 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.564253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.564256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.564281 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.566092 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.566588 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.566591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.566593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.566683 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.567004 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.567007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.567009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.567224 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.567440 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.568164 (+ 724us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.568281 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:44:27.565734 11190 raft_consensus.cc:729] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f55b38f9af584c07b5fa62521ab552ea, State: Initialized, Role: FOLLOWER
I20260806 18:44:27.569144 11190 consensus_queue.cc:260] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } }
W20260806 18:44:27.569285 11339 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:27.569356 11339 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:27.569662 11306 log.cc:824] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:27.572429 9159 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:27.572494 11190 sys_catalog.cc:564] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:27.573662 11038 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a09e162ce8d8414fadf4d947887f79e9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35392
I20260806 18:44:27.573863 11038 raft_consensus.cc:482] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:44:27.573967 11038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:27.570179 10977 master_runner.cc:186] Error getting master registration for 127.8.241.253:36693: OK, instance_id { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" instance_seqno: 1786041867283441 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:44:27.574160 10977 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:44:27.575210 11038 raft_consensus.cc:504] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } }
I20260806 18:44:27.575202 11341 sys_catalog.cc:455] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.575331 11341 sys_catalog.cc:458] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:27.576208 11038 leader_election.cc:288] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [CANDIDATE]: Term 1 election: Requested vote from peers f55b38f9af584c07b5fa62521ab552ea (127.8.241.253:36693), dc98b7e7ff9a4ada9436fec685ce847f (127.8.241.252:34435)
I20260806 18:44:27.577103 11322 sys_catalog.cc:455] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.577243 11322 sys_catalog.cc:458] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:27.577149 11155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a09e162ce8d8414fadf4d947887f79e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f55b38f9af584c07b5fa62521ab552ea"
I20260806 18:44:27.577395 11155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:27.578747 11155 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a09e162ce8d8414fadf4d947887f79e9 in term 1.
I20260806 18:44:27.579150 10997 leader_election.cc:302] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a09e162ce8d8414fadf4d947887f79e9, f55b38f9af584c07b5fa62521ab552ea; no voters:
W20260806 18:44:27.581046 11360 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 18:44:27.581084 11322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 18:44:27.581131 11360 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:27.581308 11322 raft_consensus.cc:686] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [term 1 LEADER]: Becoming Leader. State: Replica: a09e162ce8d8414fadf4d947887f79e9, State: Running, Role: LEADER
I20260806 18:44:27.581683 11322 consensus_queue.cc:237] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } }
I20260806 18:44:27.580463 11088 master_runner.cc:186] Error getting master registration for 127.8.241.252:34435: OK, instance_id { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" instance_seqno: 1786041867517415 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:44:27.583712 11088 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:44:27.583663 11322 sys_catalog.cc:455] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a09e162ce8d8414fadf4d947887f79e9. Latest consensus state: current_term: 1 leader_uuid: "a09e162ce8d8414fadf4d947887f79e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.583809 11322 sys_catalog.cc:458] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:27.584316 11370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:27.586030 11370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:27.591257 11306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f: No bootstrap required, opened a new log
I20260806 18:44:27.595557 11306 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } }
I20260806 18:44:27.595845 11306 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc98b7e7ff9a4ada9436fec685ce847f, State: Initialized, Role: FOLLOWER
I20260806 18:44:27.596230 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.585696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35408 (local address 127.8.241.254:39023)
0806 18:44:27.585812 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.585816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.586962 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 18:44:27.587265 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.587269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.587322 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.587373 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.588724 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.589303 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.590416 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.590758 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.590848 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.592621 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.592625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.592628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.592656 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.594366 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.594838 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.594842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.594843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.594928 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.595218 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.595221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.595223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.595446 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.595555 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.595932 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.596061 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:44:27.596204 11306 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } }
I20260806 18:44:27.598810 11306 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:27.598876 11263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a09e162ce8d8414fadf4d947887f79e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dc98b7e7ff9a4ada9436fec685ce847f"
I20260806 18:44:27.599133 11263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:27.600569 11263 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a09e162ce8d8414fadf4d947887f79e9 in term 1.
I20260806 18:44:27.606130 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.586230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59194 (local address 127.8.241.252:34435)
0806 18:44:27.586364 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.586369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.589927 (+ 3558us) server_negotiation.cc:407] Connection header received
0806 18:44:27.590045 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.590048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.590098 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.590171 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.594314 (+ 4143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.595027 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.597042 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.597359 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.597438 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.601944 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.601949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.601951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.601981 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.604096 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.604591 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.604595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.604597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.604685 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.605057 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.605060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.605062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.605303 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.605440 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.605854 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.605976 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:44:27.606789 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:27.585434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.8.241.253:36693)
0806 18:44:27.585561 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.585566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.596833 (+ 11267us) server_negotiation.cc:407] Connection header received
0806 18:44:27.597037 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.597041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.597097 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.597147 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.597977 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.598681 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.599673 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:27.599985 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:27.600058 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:27.602174 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:27.602178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:27.602180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:27.602469 (+ 289us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:27.604964 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.605443 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.605447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.605448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.605535 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:27.605823 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:27.605827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:27.605829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:27.606032 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:27.606141 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:27.606525 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 18:44:27.606639 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:44:27.608388 11395 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:27.608469 11395 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:27.613273 11375 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.613384 11375 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:27.617460 11155 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [term 1 FOLLOWER]: Refusing update from remote peer a09e162ce8d8414fadf4d947887f79e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:27.618172 11263 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [term 1 FOLLOWER]: Refusing update from remote peer a09e162ce8d8414fadf4d947887f79e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:27.618748 11368 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:27.618949 11322 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:27.623004 11341 sys_catalog.cc:455] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader a09e162ce8d8414fadf4d947887f79e9. Latest consensus state: current_term: 1 leader_uuid: "a09e162ce8d8414fadf4d947887f79e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.623124 11341 sys_catalog.cc:458] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:27.623435 11375 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a09e162ce8d8414fadf4d947887f79e9. Latest consensus state: current_term: 1 leader_uuid: "a09e162ce8d8414fadf4d947887f79e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.623554 11375 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:27.632143 11322 sys_catalog.cc:455] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a09e162ce8d8414fadf4d947887f79e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.632294 11322 sys_catalog.cc:458] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:27.632401 11374 mvcc.cc:205] Tried to move back new op lower bound from 7315627489752080384 to 7315627489618432000. Current Snapshot: MvccSnapshot[applied={T|T < 7315627489752080384}]
I20260806 18:44:27.633632 11375 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a09e162ce8d8414fadf4d947887f79e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.633770 11375 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:27.638195 11401 mvcc.cc:205] Tried to move back new op lower bound from 7315627489752080384 to 7315627489618432000. Current Snapshot: MvccSnapshot[applied={T|T < 7315627489752080384 or (T in {7315627489752080384})}]
I20260806 18:44:27.640367 11368 sys_catalog.cc:455] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a09e162ce8d8414fadf4d947887f79e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.640506 11368 sys_catalog.cc:458] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:27.641750 11341 sys_catalog.cc:455] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a09e162ce8d8414fadf4d947887f79e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a09e162ce8d8414fadf4d947887f79e9" member_type: VOTER last_known_addr { host: "127.8.241.254" port: 39023 } } peers { permanent_uuid: "f55b38f9af584c07b5fa62521ab552ea" member_type: VOTER last_known_addr { host: "127.8.241.253" port: 36693 } } peers { permanent_uuid: "dc98b7e7ff9a4ada9436fec685ce847f" member_type: VOTER last_known_addr { host: "127.8.241.252" port: 34435 } } }
I20260806 18:44:27.641876 11341 sys_catalog.cc:458] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:44:27.642526 11370 catalog_manager.cc:1347] Generated new cluster ID: c038b1a7b72b4dff9cfdb2e7742725ae
I20260806 18:44:27.642587 11370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:27.643174 11400 mvcc.cc:205] Tried to move back new op lower bound from 7315627489752080384 to 7315627489618432000. Current Snapshot: MvccSnapshot[applied={T|T < 7315627489752080384}]
I20260806 18:44:27.667498 11370 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:27.668954 11370 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:27.683100 11370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a09e162ce8d8414fadf4d947887f79e9: Generated new TSK 0
I20260806 18:44:27.684044 11370 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:44:27.707674 11313 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.705082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59196 (local address 127.8.241.252:34435)
0806 18:44:27.705210 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.705216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.706528 (+ 1312us) server_negotiation.cc:407] Connection header received
0806 18:44:27.706733 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.706738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.706790 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.706843 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.707525 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59196: BlockingRecv error: recv got EOF from 127.0.0.1:59196 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 18:44:27.709335 11085 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.705876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35424 (local address 127.8.241.254:39023)
0806 18:44:27.705995 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.706000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.708354 (+ 2354us) server_negotiation.cc:407] Connection header received
0806 18:44:27.708532 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.708535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.708586 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.708635 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.709187 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35424: BlockingRecv error: recv got EOF from 127.0.0.1:35424 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 18:44:27.711488 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.709976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42042 (local address 127.8.241.253:36693)
0806 18:44:27.710094 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.710098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.710572 (+ 474us) server_negotiation.cc:407] Connection header received
0806 18:44:27.710740 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.710743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.710794 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.710843 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.711346 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42042: BlockingRecv error: recv got EOF from 127.0.0.1:42042 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 18:44:27.716123 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.714652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42048 (local address 127.8.241.253:36693)
0806 18:44:27.714752 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.714756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.715183 (+ 427us) server_negotiation.cc:407] Connection header received
0806 18:44:27.715420 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.715423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.715501 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.715549 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.716025 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42048: BlockingRecv error: recv got EOF from 127.0.0.1:42048 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 18:44:27.721215 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.719776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.8.241.253:36693)
0806 18:44:27.719873 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.719876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.720330 (+ 454us) server_negotiation.cc:407] Connection header received
0806 18:44:27.720504 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.720508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.720555 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.720601 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.721106 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42052: BlockingRecv error: recv got EOF from 127.0.0.1:42052 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 18:44:27.726871 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.725437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42060 (local address 127.8.241.253:36693)
0806 18:44:27.725530 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.725534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.725987 (+ 453us) server_negotiation.cc:407] Connection header received
0806 18:44:27.726155 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.726158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.726221 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.726274 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.726766 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42060: BlockingRecv error: recv got EOF from 127.0.0.1:42060 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 18:44:27.734890 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.733279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42072 (local address 127.8.241.253:36693)
0806 18:44:27.733431 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.733435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.733942 (+ 507us) server_negotiation.cc:407] Connection header received
0806 18:44:27.734150 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.734153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.734217 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.734271 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.734782 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42072: BlockingRecv error: recv got EOF from 127.0.0.1:42072 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 18:44:27.747449 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.745666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42088 (local address 127.8.241.253:36693)
0806 18:44:27.745796 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.745801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.746382 (+ 581us) server_negotiation.cc:407] Connection header received
0806 18:44:27.746608 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.746612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.746670 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.746726 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.747325 (+ 599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42088: BlockingRecv error: recv got EOF from 127.0.0.1:42088 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 18:44:27.768393 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.766702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42092 (local address 127.8.241.253:36693)
0806 18:44:27.766825 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.766829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.767373 (+ 544us) server_negotiation.cc:407] Connection header received
0806 18:44:27.767573 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.767576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.767630 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.767678 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.768256 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42092: BlockingRecv error: recv got EOF from 127.0.0.1:42092 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 18:44:27.804397 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.803155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42098 (local address 127.8.241.253:36693)
0806 18:44:27.803286 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.803291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.803536 (+ 245us) server_negotiation.cc:407] Connection header received
0806 18:44:27.803678 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.803680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.803730 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.803801 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.804277 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42098: BlockingRecv error: recv got EOF from 127.0.0.1:42098 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 18:44:27.871662 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:27.870584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42100 (local address 127.8.241.253:36693)
0806 18:44:27.870706 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:27.870710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:27.870871 (+ 161us) server_negotiation.cc:407] Connection header received
0806 18:44:27.870999 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:27.871002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:27.871052 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:27.871132 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:27.871538 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42100: BlockingRecv error: recv got EOF from 127.0.0.1:42100 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:44:28.003224 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:28.002050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42116 (local address 127.8.241.253:36693)
0806 18:44:28.002212 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:28.002218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:28.002312 (+ 94us) server_negotiation.cc:407] Connection header received
0806 18:44:28.002499 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:28.002502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:28.002552 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:28.002624 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:28.003098 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42116: BlockingRecv error: recv got EOF from 127.0.0.1:42116 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 18:44:28.262318 11200 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:28.261252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.8.241.253:36693)
0806 18:44:28.261393 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:28.261399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:28.261469 (+ 70us) server_negotiation.cc:407] Connection header received
0806 18:44:28.261622 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:28.261625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:28.261674 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:28.261740 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:28.262196 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42124: BlockingRecv error: recv got EOF from 127.0.0.1:42124 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:44:28.582521 11360 catalog_manager.cc:1259] Loaded cluster ID: c038b1a7b72b4dff9cfdb2e7742725ae
I20260806 18:44:28.582614 11360 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea: loading cluster ID for follower catalog manager: success
I20260806 18:44:28.584291 11360 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea: acquiring CA information for follower catalog manager: success
I20260806 18:44:28.585522 11360 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f55b38f9af584c07b5fa62521ab552ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:44:28.609921 11395 catalog_manager.cc:1259] Loaded cluster ID: c038b1a7b72b4dff9cfdb2e7742725ae
I20260806 18:44:28.610013 11395 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f: loading cluster ID for follower catalog manager: success
I20260806 18:44:28.611760 11395 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f: acquiring CA information for follower catalog manager: success
I20260806 18:44:28.613032 11395 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dc98b7e7ff9a4ada9436fec685ce847f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:44:28.780259 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:28.776368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42134 (local address 127.8.241.253:36693)
0806 18:44:28.776601 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:28.776605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:28.776779 (+ 174us) server_negotiation.cc:407] Connection header received
0806 18:44:28.776928 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:28.776931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:28.777095 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:28.777202 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:28.777520 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.778045 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.778935 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.779147 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.779215 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:28.779473 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:28.779920 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:28.780009 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:28.780076 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 18:44:28.780129 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":74,"threads_started":1}
I20260806 18:44:28.786657 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:28.783334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59200 (local address 127.8.241.252:34435)
0806 18:44:28.783553 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:28.783557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:28.783569 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:44:28.783651 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:28.783654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:28.783788 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:28.783885 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:28.784048 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.784580 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.785387 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.785609 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.785680 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:28.785899 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:28.786337 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:28.786424 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:28.786452 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 18:44:28.786499 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":65,"threads_started":1}
I20260806 18:44:28.793311 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:28.789542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59204 (local address 127.8.241.252:34435)
0806 18:44:28.789832 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:28.789837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:28.790155 (+ 318us) server_negotiation.cc:407] Connection header received
0806 18:44:28.790308 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:28.790311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:28.790361 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:28.790423 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:28.790743 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.791489 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.792338 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.792573 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.792657 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:28.792894 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:28.792966 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:28.793046 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:28.793124 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 18:44:28.793174 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260806 18:44:28.794023 11507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:28.789677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42142 (local address 127.8.241.253:36693)
0806 18:44:28.789785 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:28.789789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:28.790087 (+ 298us) server_negotiation.cc:407] Connection header received
0806 18:44:28.790256 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:28.790259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:28.790318 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:28.790384 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:28.790591 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.791348 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.792724 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.793081 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.793169 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:28.793573 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:28.793653 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:28.793727 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:28.793845 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 18:44:28.793899 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:44:28.794567 11522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:28.789506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35434 (local address 127.8.241.254:39023)
0806 18:44:28.789743 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:28.789748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:28.790184 (+ 436us) server_negotiation.cc:407] Connection header received
0806 18:44:28.790305 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:28.790308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:28.790470 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:28.790580 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:28.790911 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.791427 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.792734 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:28.793113 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:28.793194 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:28.793762 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:28.794123 (+ 361us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:28.794201 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:28.794361 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:44:28.794426 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260806 18:44:28.797401 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 10977
I20260806 18:44:28.814510 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 11088
I20260806 18:44:28.825325 9159 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5RraP2/build/debug/bin/kudu with pid 11202
2026-08-06T18:44:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3174 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6379 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291064 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST